Lisa floppy drive emulation with Floppy Emu is a success! Begone, evil rotational disk drives, with your failure-prone gummed-up mechanisms. The age of SD card Lisa floppy emulation is here!

After a lot of tinkering, it’s now working on both the Lisa 2/5 and 2/10, and for both Lisa-format and Mac-format disks. I’m able to boot the Lisa from the emulated floppy drive, and use it to install Lisa Office System and other apps to the HD, and copy apps and data files from the Lisa HD to the emulated floppy drive. It also works for Mac-format disks, when running the Lisa under Macworks.

The final hurdle was realizing that the custom adapter I’d built to simultaneously connect a 400K floppy drive and a Floppy Emu was causing problems. After getting rid of the adapter and connecting the Emu directly to the Lisa, in place of the normal floppy drive, it’s now working perfectly.

 
Disk Copy 4.2 Format

From the previous Lisa firmware that I described in an earlier post, the last big piece of the puzzle was adding support for “tags”. On a Lisa floppy disk, a sector consists of 512 bytes of data and 12 bytes of tags. Raw disk image files don’t save the tag info – they’re just a byte-by-byte copy of the 512 data bytes from each sector. So in order to correctly emulate Lisa-native disks, it was necessary to use a disk image format that preserves tags: Disk Copy 4.2 images, the format I love to hate.

Floppy Emu already supported DC42 format image files for use with the Macintosh, but they were treated as read-only disks by the emulator firmware. The problem with tagged DC42 images is that the logical sector data isn’t aligned in the image file, nor in the FAT32 filesystem of the SD Card, and the sector tags are stored separately from the sector data. So to load a single sector of the emulated floppy (512 bytes data plus 12 bytes tags), it’s necessary to load three or sometimes four 512 byte sectors from the SD card, depending how the data and tags for that particular sector are aligned. It’s a pain in the butt.

Writing DC42 images is more complicated still, which is why previous firmware treated DC42 images as read-only. To perform the unaligned writes, the firmware must read a sector from the SD card, modify part of it, then write the modified version back. Again, this can involve three or four SD card sectors in order to write a single emulated floppy sector. SD cards really don’t like this pattern of read-write-read-write for single random-access sectors, so performance suffers a bit. But the new firmware code does its job, and writing to DC42 disk image files is now possible.

 
Checksums

The last unsolved item is DC42 file checksums. A DC42 image file contains a 32-bit checksum of all the data, and a separate 32-bit checksum of all the tags. If you attempt to load an image file with invalid checksums in Disk Copy 4.2, it will refuse to open it. Floppy Emu doesn’t care about the checksums, but there can be a problem if you copy a modified DC42 image file off your SD card, and try to use it with Disk Copy 4.2 or another utility that does care about checksums.

What’s the best way to make sure the checksums are kept up to date, when the DC42 image file is modified? Now that I know the checksum algorithm, I can recompute the checksum easily enough, but when should I do it?

• It’s impractical to recompute and save a whole-disk checksum, every time any sector is modified.
• I can update the checksum when the disk image is ejected, but that’s dangerous because ejection is very often a prelude to auto power-off a moment later. It takes about 2 seconds to recompute the checksum for a 400K disk, and 4 seconds for an 800K disk. Usually the time between ejection and shutdown is less than that. If the Floppy Emu loses power while it’s writing the new checksum value, the SD card could become corrupted.
• I might recompute the checksum when the disk image is inserted, but that’s unintuitive. Re-insert the disk image, in order to fix its checksums? And for systems like the Mac that write the last mount time whenever a disk is inserted, the checksum would still always be wrong.
• I might add some new Floppy Emu menu option or feature to recompute the checksum on demand. But that’s a strange special case I’d like to avoid, and doesn’t feel like the right solution.
• I could do nothing, and let people fix up the checksums with an offline tool if they ever need to copy them off the SD card and reimport them to Disk Copy 4.2. This is my solution for now.

 
Try It

If you’ve got a Lisa and a Floppy Emu, you can try the new firmware yourself. I’ve named this firmware version lisa-emu-1.0S6-F11, and it contains all the changes needed to emulate Lisa-format and Macintosh-format floppies on the Lisa computer, and to boot a Lisa from the Floppy Emu board. 1.0S6-F11 contains some changes that aren’t compatible with early Macintosh models, so while you won’t hurt anything by trying it on a Mac system, it may not work. For best results, use the normal Mac firmware for Floppy Emu with Macintosh computers, and use 1.0S6-F11 only when working with a Lisa system.

Read 9 comments and join the conversation 

Recently I launched the Sonic Bow Tie – the first project I’ve ever designed from the start to be a product, instead of a project that turned into a product by accident. The Bow Tie is an electronic kit with a circuit board shaped and sized like a real bow tie. It’s a blinking, beeping, musical tuneable theremin-type thing you can actually wear – perfect for those nerd-centric formal occasions, or project night at the local hacker space. Following the examples of the Weevil Eye, the Simon, the Drawdio, and basically everything at ThinkGeek, it’s intended to be a fun project kit to assemble with a friend, appealing to engineers with a sense of humor. I mean who wouldn’t want a dazzling neck-based audio-visual spectacle?

Unfortunately, the rest of the world doesn’t seem to agree that bow ties plus electronics is a match made in heaven. Since I launched the Sonic Bow Tie about ten days ago, it’s received zero feedback, zero interest, zero sales, zero anything. It’s enough to make me wonder – did I actually hit the “publish” button on that product announcement? Yup, I did.

This isn’t the end of the world, and my financial investment in bow tie kits is small. But what disturbs me is that something I thought would be fairly popular, or at least fairly amusing, proved to be such a complete flop. I spent many hours iterating on the circuit design and physical layout, sourcing parts, and manufacturing kits for an idea that proved to be a total non-starter. It shows that using yourself and your immediately family as a proxy for public opinion may not be the smartest idea. Meanwhile, if I don’t find more bow tie enthusiasts, I’ll be buying space in a New Mexico landfill next to all those unsold Apple Lisas and E.T. cartridges. Live and learn!

Do you have any good product failure stories you’re willing to share? Post ’em here and make me feel better. 🙂

Read 9 comments and join the conversation 

I’ve been crawling slowly closer to a working floppy emulator for the Apple Lisa, using my existing Floppy Emu hardware. I’ve now got something that’s broadly usable for emulation of Macintosh disks on the Lisa 2/10, when running under the Macworks environment, though it’s still far from perfect. The next step will be emulation of native Lisa floppy disks, so the computer can boot from the Emu and use it from within the Lisa Office System. Things are looking promising!

 
Setup

So far, most of the effort has gone into getting my borrowed Lisa to the point where I can actually run floppy tests. Initially it seemed no setup would be necessary – just plug the Floppy Emu into the Lisa, and see if it works – but reality has proven different:

• Create a hacked version of Basic Lisa Utility that doesn’t need the M key (my M key is broken)
• Buy/build a Lisa-to-PC serial cable to capture the BLU log data
• Build a custom floppy drive A/B switch, so I can connect a real 400K drive and a Floppy Emu at the same time
• Rewrite my floppy tester utility program in Think C 5.0, to avoid a mysterious crash bug on the Lisa

In order to capture the log data from BLU, I needed a serial connection to the Lisa. I ordered this USB-to-DB9 serial adapter, along with a separate DB9-to-DB25 adapter, and waited impatiently for them to arrive. After receiving the adapter, I connected the Lisa up to my Windows 7 PC, and of course it didn’t work. The link appeared totally dead, and nothing I did on one computer was seen by the other.

I’d been corresponding by email with a few Lisa experts, and one of them pointed out what I should have realized about the serial connection: I needed a null modem connection, not a straight serial connection. Serial cables have separate TXD and RXD lines for transmit and receive, and in normal usage the computer will transmit on TXD and receive on RXD, while the peripheral will receive on TXD and transmit on RXD. But with my Lisa-to-Windows hookup, both ends thought they were “the computer”, so both were trying to transmit on TXD. Argh!

I looked into buying a null modem adapter, which would also have required getting a gender changer too. But I got impatient, and finally just built my own solution out of some jumper wires to swap TXD and RXD. It works.

 
Emulation

The best diagnostic tool I’ve found is the floppy tester function of BLU, which generates sector-by-sector debug info. There are some categories of disk problems that BLU doesn’t report, such as drive speed adjustment problems and synchronization problems, but it’s still the most powerful tool for the job.

I’ve been corresponding with two people who own Lisas and Floppy Emus, and the three of us have been testing various Emu firmware versions and comparing the test results. With the latest “Lisa Emu” firmware, the emulation doesn’t work at all on a Lisa 2/5, but does work on a 2/10. These two Lisa models have completely different floppy controller hardware that should be functionally equivalent, but some hidden difference is clearly important here. More on that later.

On the Lisa 2/10, we discovered that the Emu firmware routine that generates the 10-bit bitslip marker wasn’t working. The bitslip marker is used by the floppy controller to find the correct byte-to-byte framing in the floppy data serial bitstream. After fixing that, the emulation started worked on the Lisa, but very slowly. There were clearly still lots of floppy errors happening, slowing down the I/O, even though it eventually succeeded.

 
Disk Rotation Speed

When testing with BLU, doing a sequential read of all the sectors on the disk, it appeared to freeze for about 15 seconds before reading the first sector on each track. BLU didn’t report any errors, but there was clearly something wrong. After some discussion with my remote Lisa testing partners, we began to suspect this was a problem with the simulated rotation speed of the emulated disk, and we were right.

With an Apple 400K floppy disk drive, the drive spins at different speeds depending on which track is being accessed. The disk is divided into five speed zones, with five different rotational speeds ranging from 394 to 590 RPM. The computer directly controls the drive’s rotational speed by modulating a signal called PWM, and the drive indicates its current speed with a signal called TACH. So the computer sets PWM, then reads TACH and verifies that the desired rotational speed has been reached. If the verification fails, a Macintosh will report error -79: “can’t correctly adjust disk speed”. But what will a Lisa do?

The Floppy Emu doesn’t actually use the PWM signal, but instead it sets the TACH value directly, based upon which track is accessed by the computer. I tried modifying the Emu firmware to generate an obviously wrong TACH, and the behavior on the Lisa was unchanged. It still wasn’t reporting any errors, but the 15 second pause at the start of each track was still there. This told me that the TACH value was probably wrong all along, and that the Lisa was waiting about 15 seconds on each track for the TACH speed to become correct, then giving up and attempting the I/O anyway.

I blindly experimented with different formulas for TACH. Make it faster? Slower? Change the speed by a percentage, or by a constant offset? After lots of trial and error, I found that scaling TACH by 2.5% across the board made all the 15 second pauses disappear. So in effect, the Emu is reporting that the drive’s rotational speed is 2.5% faster than the spec. Why is this necessary? I honestly have no idea. The revised formula is almost certainly still wrong, but it seems to work well enough for now.

 
Inter-Sector Gap and Interleave

The sectors on a normal Macintosh disk are interleaved 2:1, with a short gap of dead space between sectors. Conceptually it looks something like this timeline view:

0000000000.6666666666.1111111111.7777777777.2222222222.8888888888 etc.

The 2:1 interleave means sector 6 follows sector 0, then come sectors 1, 7, 2, 8, 3, etc. The gap between each sector is about 10% the duration of the sector itself. The purpose of the interleave is to get the fastest possible I/O speeds when doing sequential reads of many contiguous sectors, allowing for some amount of CPU processing time after reading each sector that wouldn’t be possible with a 1:1 interleave.

For a sequential read of many sectors and a correctly tuned interleave, after locating the first sector on a track, the floppy controller should see zero unwanted sectors go by before the next desired sector appears. But I discovered that for BLU’s sequential read test, the floppy controller was almost always seeing 11 unwanted sectors before the desired sector appeared. There are 12 sectors per track on this region of the floppy, so 11 unwanted sectors meant the Lisa was just missing the desired sector, and had to wait for an entire rotation of the simulated disk (11 more sectors) before the desired sector rolled around again. In short, this meant it needed a higher interleave than 2:1. But that couldn’t be right, because real Mac disks are interleaved 2:1, and the Lisa under Macworks can read them without problems.

I wasted a lot of time experimenting with different interleave values, and different gap sizes between the sectors. Eventually I found a combination that led to zero unwanted sectors after the first sector of the track, but it required a huge and unrealistic inter-sector gap size. And while it made the BLU floppy test results look better, it actually performed worse in real-world tests, copying files under Macworks.

Finally I had the idea to repeat the BLU floppy test with a real floppy disk and drive, and observed the same 11 unwanted sectors before the desired sector. Aha! This told me the issue was actually with the BLU test, and not with the Floppy Emu firmware. BLU must be doing a non-trivial amount of CPU computation or other I/O after each sector, more than is normally performed by the OS during a sequential read, so that by the time it’s ready for the next sector, the desired sector has already rotated past. I reverted all of my changes, and put the inter-sector gap size back where it was originally. The BLU results got worse again, but real-world performance under Macworks improved noticeably.

 
Recalibrations?

One more mystery remains unsolved. In all of my BLU tests, the reported value for recalibrations (Rcl) is always 4C. For one of the other testers, it’s always 02. I’m not sure yet what the third tester is seeing for Rcl. The BLU manual appendix D says Rcl is “recalibrations remaining”, and that an operation normally starts with 1 recalibration remaining and counts down from there – so both 4C and 02 are anomalous. What’s the significance of this? It needs more investigation.

 
Real-World Tests

Using a Lisa 2/10 system, I booted Macworks XL from the Lisa’s hard drive. By holding down the left option key, I told Macworks to initialize the Macintosh environment, but wait for a floppy disk to actually boot the Mac OS. Using a real 400K floppy drive and disk, I booted System 3.2, measuring the time from disk insertion to ready desktop at 37 seconds. Then I repeated the same experiment with Floppy Emu and a disk image of System 3.2, and measured it also at 37 seconds. So it works! No more mysterious slowdowns: Floppy Emu on the Lisa performs at the same speed as a real floppy.

As a second test, I booted Macworks from the Lisa’s hard drive, then used Floppy Emu to insert a 400K disk image containing MacPaint and MacWrite. The disk image appeared on the desktop within a few seconds of inserting it, and copying MacPaint from the Emu to the Lisa’s HD only took a few seconds more. I was also able to copy files from the HD back to the emulated floppy disk without problems. Given what I’m seeing in these tests, the current firmware should provide a fully usable and full-speed floppy emulation for the Lisa under the Macworks environment.

 
Try It

If you’ve got a Lisa and a Floppy Emu, you can try the new firmware yourself. I’ve named this firmware version lisa-emu-1.0S3-F11, and it contains all the fixes and adjustments described above in order to make Macintosh disk emulation work on the Lisa in the Macworks environment. Native Lisa floppy emulation still isn’t implemented, but I hope to tackle that soon.

Be the first to comment!