Here’s another big Macintosh firmware update for the BMOW Floppy Emu disk emulator, and it’s got some good stuff! Writable MOOF images, in-emulator disk formatting, performance improvements, and a significant bug fix. If you’re the impatient type, skip the rest of this text and download the new firmware now from the Floppy Emu project page. And as always, you can also buy a new Floppy Emu Model C at the BMOW Store. Or just sit back and enjoy this completely gratuitous screenshot of a logic analyzer capturing some disk operation.

 
Writable MOOFs

The 221201M update introduced support for MOOF disk images, a new image format that’s designed to capture all the low-level disk information needed for copy-protected software. Today’s update extends MOOF support by adding write capability, instead of treating MOOFs as read-only, which is essential for a few software titles that require writing to the boot disk. Print Shop, Carmen Sandiego… I’m looking at you. This sounds like a small change, but turned into a major effort to get it working reliably.

If you need to format a MOOF (see below), I recommend using the Blank 400K.moof and Blank 800K.moof disk images that are provided in the firmware’s Examples subdirectory, rather than using any random MOOF. The provided blank images have been tweaked to guarantee every track has a number of bits that’s an exact multiple of eight. This results in the best possible SD Card I/O performance, and reduces the chance of a formatting failure.

In the weeks since MOOF support was first introduced, the Moof-a-Day collection at archive.org has also grown from 52 to 78 titles, so check them out!

 
In-System Disk Formatting

All the supported disk image types can now be formatted directly on the Floppy Emu, from the attached Mac or Lisa. MOOF, DiskCopy 4.2, and raw images can all be formatted. In-system formatting normally isn’t necessary – you can usually use a pre-made blank disk image instead. But it’s essential for a few software titles like Seven Cities of Gold that require formatting a save disk directly from within the game. It’s also helpful when using disk copy tools to copy a real disk to a disk image, because most disk copy tools will automatically attempt to format the destination disk.

This is a big change, and the amount of effort that it required is way out of proportion with the value of the new feature, but I stubbornly wanted it anyway. In-system formatting has been a sore point for the Floppy Emu since the very beginning, and for a long time I believed it was impossible for technical reasons, but the work on MOOF writes helped me to realize that was only half-true.

To quote Aladdin’s Robin Williams quoting William F. Buckley, in-system formatting comes with a few, uh, provisos, a couple of quid pro quos. Here they are.

DiskCopy 4.2 and raw disk images can be formatted with a standard format, using standard address marks and data marks, standard sector sizes, standard checksumming… you get the idea. A DC42 or raw disk image format will fail if you attempt to create a non-standard disk format, like a disk that uses D5 18 3F to mark the start of each sector instead of the standard D5 AA 96. That’s because these disk image types don’t actually contain any of this format information, and it’s just implied to use standard defaults. I haven’t yet found any example where this is an issue, but if it ever comes up, formatting a MOOF disk image will allow for esoteric format types that intentionally break the standard.

For those who are paying very close attention, there’s also a subtle difference between 400K/800K formatting and 1440K formatting. The 400K/800K format is a true format, where every sector is written as zeroes, and then a new file allocation table and root directory are created. To employ a more modern term, it’s like doing a secure erase of the disk image. The 1440K format is more like doing a quick erase – it creates a new FAT and root directory, but doesn’t actually modify any of the sectors except the ones in the FAT. The Floppy Emu fools the Mac into thinking it zeroed all the sectors, but the old data is still hiding in there, which you can see if you view the disk image with a hex editor.

Formatting performance may vary depending on your SD card’s write cache behavior. If the format fails, try again or use a different SD card.

Disk images must be formatted as the same size as they were before formatting. When the Mac asks if you wish to format a disk as one-sided or two-sided, choose wisely. Attempting to reformat a 400K disk image as an 800K disk will lead to disappointment.

As mentioned earlier, for best performance when formatting a MOOF disk image, use the blank MOOF images provided with the firmware. These have been tweaked for optimal I/O speed.

 
1440K Disk Emulation Fixes

If you’ve been using 1440K disk emulation before now, I apologize. It’s been semi-broken for years, and I didn’t even realize it. I discovered there was a bug that could sometimes introduce invalid MFM clock bits into the bitstream, causing the Mac’s disk controller to freak out and think an error occurred. In the best case this substantially degraded the speed of disk transfers, and in the worst case disk transfers would fail outright with a message like “a disk error occurred”. After fixing this bug, 1440K disk emulation is dramatically better. It’s a very noticeable difference.

This is a short section with only a single paragraph, so I’ll repeat myself once more. This small fix makes a great difference, and if you use 1440K disk emulation very often then you’ll want this.

 
Performance Optimizations

The MOOF pseudo-random number generator has been greatly improved, and is used to create so-called fake bits or weak bits in certain MOOF disk images. The old random generator was pretty lousy, because I didn’t realize it might be important to get a specific profile of random bits. The bits need to mimic the random-seeming behavior of a real disk drive’s read head when no magnetic flux is present. The poor-quality randomness may have caused problems for some MOOF disks that employ copy-protection based on weak bits. The new pseudo-random number generator is a 16-bit linear feedback shift register, with a cycle length of 65535 that’s about the length of a whole track, and with a post-processing step to ensure that only 30 percent of the random bits are 1s. To maintain real-time bit streaming, the code must be optimized to generate each new random bit in less than two microseconds, along with other housekeeping tasks.

To accelerate data transfers from/to the SD card, Floppy Emu will now ignore the tag bytes contained in DiskCopy 4.2 disk images while it’s in Macintosh floppy emulation mode. The Mac doesn’t use these tag bytes anyway, so they’re just a waste of time. Lisa floppy emulation mode still supports tag bytes, as they’re an essential part of the Lisa filesystem. If you were doing something non-standard involving tag bytes on the Mac with Floppy Emu and DiskCopy 4.2 images, and this firmware update causes you problems, let’s talk.

SD transfers have also been optimized in other ways. If the Floppy Emu is in the midst of loading a track’s worth of data from the SD card, and it suddenly discovers it no longer needs that track because the head stepped to a different track, it will now abort the transfer ASAP instead of finishing the whole transfer and then discarding it. This actually happens a lot. The result is there’s less time on average between when the head steps to a track and the data for that track begins streaming from the Floppy Emu, which helps reduce the chances of disk I/O hiccups.

The raw SPI transfers from/to the SD card have also been substantially accelerated. The actual SPI clock hasn’t changed, and is still 10 MHz, but the dead time between each byte and the next one has been reduced. What dead time, you ask? The microcontroller in the Floppy Emu unfortunately doesn’t have any SPI buffering, so the CPU needs to intervene after every byte to queue up the next byte and start it, which can badly hamstring performance. With a core clock of 20 MHz and an SPI clock of 10 MHz, it should theoretically be possible to transfer a bit every 2 core clock cycles and a full byte every 16 clock cycles. But with a typical SPI polling loop like the one in the SD library I used, it needed 26 clock cycles instead of 16, only about 60 percent of the theoretical best performance. Not great.

I rewrote the innermost loop to do a blind transfer of each byte instead of polling for completion, employing some cycle-counting to ensure that each new byte would be started at exactly the right time. This improved performance to 18 clock cycles per byte, about 90 percent of the theoretical max. The loop could easily be timed for 17 or 16-cycle transfers, but the SPI hardware fails at this speed. I suspect the hardware needs 1 SPI clock (two core clocks) to reset its internal state before the start of the next byte, but the datasheet mentions nothing about this.

I hope you enjoy all these new features and improvements. That’s it for me until sometime next year. Happy holidays!

Download the latest Floppy Emu firmware from the project page.

I need a QA team with a suite of regression tests! The BMOW Floppy Emu disk emulator supports many different emulation modes, disk image types, and usage scenarios, so it’s easy to break something without realizing it. If I break something big and obvious that escapes my testing, people will complain and let me know there’s a bug. But if I break something more subtle, I might not realize it for a long while. A very long while.

Today I discovered that writing to a 1440K Macintosh disk image has been very buggy since 2019. It sort of half worked, but it was very slow because the Mac was doing a huge number of retries for each sector write. If you tried copying a large file onto a 1440K disk, it would fail about 20 percent of the time with “a disk error occurred”. After the December 1 firmware update that introduced MOOF support, the situation got even worse, and attempts to copy a large file onto a 1440K disk would almost always fail. But nobody ever reported a problem, and I didn’t realize anything was wrong.

Thank God for revision control systems. I was able to walk backwards through a huge number of past changes in the firmware code, attempting to isolate where the bug first occurred. This was incredibly tedious because prior to the recent MOOF changes, the 1440K writes sort-of-worked well enough that I needed to try many separate file copies and count how many succeeded and failed before I could be sure whether the bug was present. It took about 10-15 minutes per changelist, and there were a LOT of changelists. The whole process filled nearly an entire day.

And the guilty change was: OLED dimming. This change was made in May 2019. To prevent screen burn-in on the OLED display, the firmware dims the brightness after 30 seconds of inactivity. There’s a bit of code that happens after every sector that says “Has the dimming timer overflowed? Has it been 30 seconds yet?” This code was just slow enough to break the Floppy Emu’s real-time guarantee and introduce a burp into the 1440K MFM bitstream between each sector and the next one whenever the timer overflowed. While recently adding MOOF support to the firmware, I switched the dimming logic to use a different hardware timer so that I could use the original timer for MOOF purposes, and this exacerbated the MFM burp problem. The burp caused sector verifications to frequently fail after writing to the disk, resulting in a huge number of errors and retries.

Honestly I’m not sure why the burp is even an issue, since there are sync bytes before each sector that should resynchronize the Mac’s disk controller if it was thrown off by the burp. But it clearly is a problem. I made a simple change to skip the check for dimming while the disk motor is on, and it’s a night and day difference. Copying files to 1440K disks is not only 100 percent reliable now, but it’s also about twice as fast as before. Look for a new firmware release with this fix and other changes soon.

Most people believe that modern compilers generate better-optimized assembly code than humans, but look at this example from AVR-GCC 5.4.0 with -O2 optimization level:

    7b96:	10 92 34 37 	sts	0x3734, r1	; 0x803734 <tachFlutter>
    7b9a:	e0 e0       	ldi	r30, 0x00	; 0
    7b9c:	f0 e0       	ldi	r31, 0x00	; 0
    7b9e:	a0 91 35 37 	lds	r26, 0x3735	; 0x803735 <driveTachHalfPeriod>
    7ba2:	b0 91 36 37 	lds	r27, 0x3736	; 0x803736 <driveTachHalfPeriod+0x1>
    7ba6:	ae 1b       	sub	r26, r30
    7ba8:	bf 0b       	sbc	r27, r31
    7baa:	b0 93 89 00 	sts	0x0089, r27	; 0x800089 <OCR1AH>
    7bae:	a0 93 88 00 	sts	0x0088, r26	; 0x800088 <OCR1AL>
    7bb2:	10 92 95 00 	sts	0x0095, r1	; 0x800095 <TCNT3H>
    7bb6:	10 92 94 00 	sts	0x0094, r1	; 0x800094 <TCNT3L>
    7bba:	32 2d       	mov	r19, r2
    7bbc:	e0 e0       	ldi	r30, 0x00	; 0
    7bbe:	f0 e0       	ldi	r31, 0x00	; 0
    7bc0:	f0 93 e3 33 	sts	0x33E3, r31	; 0x8033e3 <currentTrackBytePos+0x1>
    7bc4:	e0 93 e2 33 	sts	0x33E2, r30	; 0x8033e2 <currentTrackBytePos>

This is straight-line code with no branching. All registers and memory references are 8-bit. With AVR-GCC, the register r1 always holds the value 0, so the code is doing this: Set tachFlutter to 0, load driveTachHalfPeriod, set OCR1A to driveTachHalfPeriod minus 0, set TCNT3 to 0, set currentTrackBytePos to 0. There’s also a move of r2 to r19, which is used later, and I’m not sure why the compiler located the instruction here. There are at least three glaring inefficiences:

• the compiler wastes time loading 0 into r30 and r31, when it could have just used r1
• it does this TWICE, when we know r30 and r31 were already zero after the first time
• it subtracts a constant 0 from driveTachHalfPeriod

I can maybe understand the subtraction of constant 0, if there’s another code path that jumps to 7ba6 where the value in r30:r31 isn’t 0. But why wouldn’t the compiler make a completely separate path for that, with faster execution speed when the subtracted value is known to be 0, even if the code size is greater? After all this is -O2, not -Os.

It also appears there’s no optimization for setting multi-byte variables like currentTrackBytePos to zero. Instead of just storing r1 twice for the low and high bytes, the compiler first creates an unnamed 16-bit temporary variable in r30:r31 and sets its value to 0, then stores the unnamed variable at currentTrackBytePos.

This whole block of code could easily be rewritten:

    sts	0x3734, r1	; 0x803734 <tachFlutter>
    lds	r26, 0x3736	; 0x803736 <driveTachHalfPeriod+0x1>
    sts	0x0089, r26	; 0x800089 <OCR1AH>
    lds	r26, 0x3735	; 0x803735 <driveTachHalfPeriod>
    sts	0x0088, r26	; 0x800088 <OCR1AL>
    sts	0x0095, r1	; 0x800095 <TCNT3H>
    sts	0x0094, r1	; 0x800094 <TCNT3L>
    mov	r19, r2
    sts	0x33E3, r1	; 0x8033e3 <currentTrackBytePos+0x1>
    sts	0x33E2, r1	; 0x8033e2 <currentTrackBytePos>

This is much shorter, and avoids using r27, r30, and r31, so there are more free registers available for other purposes.

In the classic Macintosh world, 400K and 800K floppy disks use GCR encoding but 1440K disks use MFM encoding. The BMOW Floppy Emu disk emulator has supported 1440K MFM disk emulation for years, and I thought I knew everything about how it worked. Recently I dug into Mac MFM again, while adding support for in-emulator formatting of disk images, and discovered some bad assumptions as well as some critical but undocumented behaviors of the Macintosh-Sony high-density floppy drive – or at least behaviors not documented anywhere that I can see.

I’ll gloss over most of the errors and misunderstandings in my MFM design, but there were many of them. One of the biggest was using the wrong sync byte value in some places: $AA instead of $4E. Fixing this was surprisingly difficult, due to a design practice of repeating a single $A nibble value to create the $AA sync byte, which wouldn’t work for $4E.

 
Index Pulses

The most interesting discoveries were related to how the Mac-Sony HD floppy drive handles index pulses. Yes that’s right, index pulses. If you’re like me, you may have thought that Macintosh floppy disks are all soft-sectored and don’t use index holes. That’s true, the disks don’t have any hard indexing mechanism, but the high density drive does. For every rotation of the disk, the drive’s index signal pulses briefly high.

But where and how can the computer read this index signal? Some readers may know that 3.5 inch Sony floppy drives have 16 internal 1-bit state registers, whose values can be read by the computer by selecting a register using the CA2, CA1, CA0, and SELECT I/O signals. This table is from Inside Macintosh Volume III:

You’ll find very similar tables in a few other sources, like the IWM documentation for Linux on Mac 68000. The Floppy Emu’s 3.5 inch drive emulation is built around this table – it emulates these registers. There are registers for things like motor on, write protection, and step direction… but I don’t see anything about index pulses.

Well, it turns out that this table is only valid for 400K and 800K floppy drives, but I’ll be damned if I can find a good source that documents the differences for other types of drives. Based on some old cryptic notes, and a logic analyzer trace of the disk I/O signals while a Macintosh LC formats a real 1440K floppy, I was able to conclude that the 0111 TACH register is actually an INDEX register for 3.5 inch high density Sony drives operating in HD MFM mode. I think I may have seen documentation of this somewhere in the past, but can’t locate it now. Floppy Emu doesn’t emulate this INDEX behavior at all, and it’s not needed when reading or writing a disk, but it turns out that it’s crucial when formatting a disk.

 
Reading While Writing

While examining the logic analyzer trace from the Macintosh LC as it was formatting a disk, I noticed something else curious. The computer sometimes asserted the write-enable signal for much longer than a single rotation of the disk, which is all that should be needed for a single track. Sometimes it was almost two rotations of the disk. I also noticed an odd pattern reading the RDDATA0 and RDDATA1 registers while writing to the disk. What should the read data even be, in the middle of a write operation? I’d never thought about it before, and don’t know of any software that might attempt to do it. But I noticed there was always a single 0 to 1 transition on RDDATA somewhere in the middle of the write, occurring at different times after the start of the write for each track, but always appearing about 200 ms before the end of writing.

For a very long time, I simply wrote this off as an unimportant detail, but eventually I realized that the step transition on RDDATA while writing is the INDEX pulse. I’ve never seen this behavior documented anywhere, but after reviewing the logic analyzer trace it was undeniable. The computer begins to format a track by writing a continuous string of sync bytes, while simultaneously reading RDDATA0 or RDDATA1. When it sees a low to high transition there, it knows it’s the index pulse, and it begins writing the first sector beginning just after that spot. From there it continues to write all the other sectors, ending the write operation just before the index pulse is due to appear again. Finally it checks the index pulse again, this time by reading the INDEX register the normal way at 0111. If everything checks out OK, then it steps to the next track and repeats the process.

This was my ah-ha moment, when all the pieces suddenly clicked into place and I could understand why the computer was responding differently to the Floppy Emu during formatting than it did to a real drive. Now I need to make additional firmware modifications, and try to get Floppy Emu to emulate this same behavior.

Good news! Today’s BMOW Floppy Emu firmware update brings support for the Macintosh MOOF disk image format. MOOF is a new disk image format for 3.5 inch Macintosh floppy disks, designed by John K. Morris, with the goal of capturing all the low-level disk information needed for copy-protected software. It’s the Macintosh equivalent of a WOZ disk image for Apple II computers. With a MOOF, you can use original floppy disks exactly as they came from the publisher, with copy protection still intact, instead of relying on cracked or modified versions. Although media-based copy protection was never as common in the Mac world as it was for Apple IIs, there’s still a good amount of early software from the era of the Mac 128K, 512K, and Plus that’s copy-protected. Most of them are 400K disks, but there are some 800K ones too. Archive.org has a collection of MOOFs at https://archive.org/details/moofaday

 
Understanding MOOF

A regular Macintosh .dsk or .img disk image is a byte-level representation of the logical data contained in a disk. It’s all of the disk’s sector data concatenated together into one big file. It supports normal disk behavior that’s built around a typical ReadSector() and WriteSector() API. For an 800K floppy disk, the .dsk disk image is precisely 800K in size.

A MOOF disk image is a bit-level representation of the disk as it’s physically implemented in the floppy media. It’s everything on the physical disk, including the empty space between sectors, the headers and footers, and GCR 6&2 encoded data or whatever non-standard encoding the developer may have chosen. For a MOOF that’s created using John’s Applesauce hardware, it will also correctly preserve the relative angular positions of adjacent disk tracks, and areas of the disk where there are no bits, both of which are essential for some copy-protection schemes. For an 800K floppy disk, the MOOF image is typically about 1300K in size.

I don’t recommend rushing to re-encode all your Mac disks as MOOF. For the vast majority of disks, this is not only unnecessary, but counterproductive since MOOF images are larger and more difficult to use with common software tools. But for those few Mac disks with embedded media-based protection, a low-level MOOF image is just what’s needed.

 
MOOFing the Floppy Emu

When people asked me about MOOF in the past, I said it was impossible for Floppy Emu to support bit-level 3.5 inch disk images because there wasn’t enough RAM to store a complete track at the bit level. So what kind of black magic makes this possible now?

To make the data fit, the Floppy Emu firmware must depart slightly from the behavior of a real floppy disk. For a real 800K disk, there’s a small delay when switching between tracks, but switching between sides is instantaneous. For a normal .dsk disk image, the Emu can support this by storing track 0 side 0 and track 0 side 1 in RAM at the same time. There’s only a delay when new data is fetched from the SD card for the next track, which mimics the delay of a real floppy drive’s mechanical head stepping movement. But for 800K MOOF images, there isn’t enough RAM to store both sides. Only track 0 side 0 is stored in RAM and there’s a delay for SD card access when switching to track 0 side 1. In theory this might cause software errors or copy-protection failures, since there’s no equivalent delay with a real disk. But in practice, I don’t think it’s a problem.

This RAM-saving trick works for 400K and 800K MOOFs, but can only be extended so far. RAM limits are one of several reasons you’ll probably never see 1440K MOOF support on this hardware. But that’s no great loss since the era of media-based copy protection had mostly ended by the time 1440K disks were introduced. All of the MOOF examples in Archive.org’s Moof-A-Day collection are 400K and 800K disks.

This MOOF implementation for Floppy Emu is read-only. A few Moof-A-Day games require writing to the disk in order to play, so although they load and run, they’re not fully usable. A writable version of MOOF support is in the works. The rare MOOFs that use pure FLUX data are unsupported by the Floppy Emu, due to hardware limitations. The only example that I know is the game OIDS.

 
WOZ 3.5

The MOOF format is essentially identical to the 3.5 inch WOZ format for Apple II computers, just with different metadata. That means it’s theoretically possible to add 3.5 inch WOZ support to Floppy Emu too, and that’s my hope. But it’s unfortunately not as simple as just re-using the Macintosh MOOF code: the Apple II version of the firmware is substantially more complex than the Mac version, because it supports many more types of disk emulation and disk image formats, and it’s already pushing the limits of the microcontroller’s performance and memory, and the CPLD logic chip’s resources. It’ll be a major challenge to weave 3.5 inch WOZ support into this environment.

 
Moof-A-Day Known Issues

• Two titles still don’t pass the copy-protection check, for unknown reasons: The Ancient Art of War and The Surgeon. The Surgeon also fails its check with archive.org’s built-in MAME environment, so it’s not just Floppy Emu that has trouble with it.
• Mac Vegas will silently fail to write to the read-only disk, then restart or crash when you begin a mini-game.
• Some games have other requirements like only running properly on a Mac 512K, or only when the disk is in the internal drive, or they require two disk drives.

Please give this a spin, and let me know of any problems you find with MOOF support, or with other disk emulation features that might have broken accidentally.

Download the latest Floppy Emu firmware from the project page, or buy a new Floppy Emu Model C at the BMOW Store.