Floppy Emu Has Cholera
Achievement unlocked! 5.25 inch 140K disk emulation for Apple II is now working for Floppy Emu. I booted up my Apple IIe with an Oregon Trail disk image, lost some oxen, got cholera, and died. The 5.25 inch disk emulation has been tested on a IIgs with the built-in floppy port, on a IIe with the 19-pin Disk 5.25 controller card, and on a IIe with the classic Disk II controller card. Awesome!
This brings the total number of disk formats that Floppy Emu can emulate to eight: Mac 400K, Mac 800K, Mac 1.44MB, Mac hard drive (HD20 compatible), Lisa 400K, Lisa 800K, Apple II 800K (3.5 inch), and Apple II 140K (5.25 inch).
These are early days for the 5.25 inch Apple II disk emulation, so I won’t be posting the new firmware quite yet. There are still a number of issues remaining that I hope to resolve soon. The big one is write support: the current firmware doesn’t yet implement 5.25 inch write emulation, so disks are read-only. The disk image format support also isn’t complete, and raw DOS 3.3 order images (.do files) are the only format that works. I expect to add ProDOS order (.po files) and 140K 2MG images soon – or is 2MG only used for 800K disk images? I’ll have to check.
In my previous post, I mentioned that the raw disk bytes weren’t always being received correctly by the Apple II. It turned out that the width of my “1” pulse was too narrow, and sometimes a 1 would be detected as a 0. Data arrives from the disk at a rate of 4 microseconds per bit, where a logical 0 is sent as 4 microseconds of 0 volts, and a logical 1 is sent as a short pulse to 5V and back within a 4 microsecond window that’s otherwise 0 volts. Initially the width of my pulse was 0.25 microseconds. When I widened it to 0.75 microseconds, it started working reliably. That’s curious – is the disk controller actually doing polling, instead of edge triggering? Why didn’t the narrower pulse work?
It was a headache getting the checksum algorithm working correctly. The checksum is described in the book Beneath Apple DOS, but even after re-reading it several times, I was still confused. The algorithm is tightly coupled to the method the Apple II uses to read and write disk data, where it performs a process called “pre-nibbilization” before writing each sector, separating the upper and lower bits of each data byte and storing them in two different buffers. Floppy Emu works differently, performing nibbilization on the fly, so it was somewhat awkward to transmit data and checksums the same way.
Interleave and Sector Ordering
The other question that I struggled with was sector ordering. If you’ve been around the Apple II emulation world for a while, you’ve probably heard the terms “DOS order” and “ProDOS order” when referring to disk images. You may have also seen some discussion of sector interleaving within a track. The meaning of these terms and the interplay between them took me quite a while to understand.
Some background: each sector on the disk is prefaced with a sector header. The header is not part of the sector itself, and doesn’t count as “data” when you’re totalling up how much data a disk contains. A crucial piece of information contained in the header is the sector number, used by the operating system to identify which sector is which. Typically the sectors aren’t actually stored consecutively on the floppy disk, but are interleaved. A Macintosh 800K floppy track has the sectors interleaved like this:
0 6 1 7 2 8 3 9 4 10 5 11
This is a performance optimization. The floppy disk is always spinning, and without interleaving, a vintage computer would be too slow to finish processing the data from sector 0 before sector 1 spun under the head. By the time it was ready to begin reading sector 1, that sector would have already spun past, so the computer would have to wait for it to spin all the way around again. By staggering the sectors on the floppy disk, the computer gains some extra breathing room and avoids needing a full rotation for the next sector.
Things begin to get confusing when we talk about physical sectors vs logical sectors. Physical sector numbers are the numbers contained in the sector headers on the disk. Logical sector numbers are determined by the OS, and define the data contained in each sector. If you stored a large text file on a floppy disk, the first few sentences would be stored in logical sector 0, the next few in logical sector 1, then logical sector 2, etc.
On the Mac, physical and logical sector numbers are the same. The sector number in the sector header is the same as the number assigned by the Mac OS. If we use A[B] to describe a sector whose sector header number is A and whose data contents are the operating system’s sector number B, then the Mac has a 1:1 relationship:
0[0] 6[6] 1[1] 7[7] 2[2] 8[8] 3[3] 9[9] 4[4] 10[10] 5[5] 11[11]
The Apple II handles interleaving differently. Physical sector numbers aren’t interleaved, and instead they’re just numbered consecutively 1 to N along the track. But the operating system uses so-called “software skewing”, where logical sector N is not necessarily stored in physical sector N. For DOS 3.3, the relationship looks like this:
0[0] 1[7] 2[15] 3[6] 4[14] 5[5] 6[12] 7[4] 8[11] 9[3] 10[10] 11[2] 12[9] 13[1] 14[8] 15[15]
So when a DOS 3.3 Apple II program wants sector 4, it actually fetches physical sector 7 from the floppy disk. For all practical purposes, physical sector 7 is sector 4. My head hurts already.
With ProDOS, logical data is stored in 512 byte blocks instead of 256 byte sectors. Each 512 byte ProDOS block N is stored in two separate 256 byte physical sectors, NA and NB:
0[0A] 1[4A] 2[0B] 3[4B] 4[1A] 5[5A] 6[1B] 7[5B] 8[2A] 9[6A] 10[2B] 11[6B] 12[3A] 13[7A] 14[3B] 15[7B]
Confused yet? A DOS ordered disk image is one in which the physical sectors are stored in ascending order by DOS 3.3 logical sector number. A ProDOS ordered disk image is one in which the physical sectors are stored in ascending order by ProDOS logical block number. While the two orderings are obviously derived from the two different operating systems, they are ultimately just arbitrary ways of grouping data into 256 byte chunks. DOS 3.3 software can be stored in ProDOS ordered disk images, and vice-versa.
11 Comments so far
Leave a reply. For customer support issues, please use the Customer Support link instead of writing comments.
Looks like there CAN be a 140k 2mg file. I think 2mg is specifically design to be universal (for emulators?) so it contains either a DO or PO image, plus some catalog data (size,format)
Looks like this FTP has 140k 2mg files though for you to play with.
http://www.brutaldeluxe.fr/products/underground/the%20brain%20trust/
There are also various 140k 2mg files on ASIMOV
Hi Steve, is your emulation of 5.25″ floppies on the Apple ][ also designed to work with nonstandard/custom sector storage formats (e.g. RWTS18?) Some “copy protected” disks packed sectors more tightly so that disk copying programs of the day didn’t know how to handle them– I hope Floppy Emu can emulate those disks directly rather than relying on “cracked” versions or workarounds.
Is there a standard image file format for such copy-protected disk images? Do you have a link with more info?
Only digital format I know of that captures the contents of a RWTS18 disk is the Kryoflux raw STREAM format (possibly also DRAFT). Raw Kryoflux files can be pretty huge though (upwards of over 1GB for a single disk), so it may not be practical. The RWTS18 format is described here http://www.textfiles.com/apple/CRACKING/tangled.txt
Supposedly, Saltine’s Super Transcopy can create images of copy-protected disks as .NIB files, but I don’t have any experience with either the program or the format.
The most popular protected image format readable (not always writable) by emulators is the nibblized disk image (.NIB), which didn’t 100% replicate all copy protected disks. Thing is that was the best one could get until devices like the Kryoflux appeared and could 100% replicate a copy protected disk bit by bit. EDD and FDI appear to be some of these formats although I can’t say I have encountered them (there is only one EDD format image on the popular asimov archive at the moment!).
http://fileformats.archiveteam.org/wiki/EDD_disk_image
http://www.brutaldeluxe.fr/products/apple2/imfEDDup/
http://www.oldskool.org/disk2fdi/files/FDISPEC.pdf
My goal is to get all the permutations of normal 140K Apple II disk images working, including .NIB, which is a raw dump of the nibbilized track data. I think that should be enough to cover 99% of the software out there from Apple II archives like Asimov. When that’s done, I’ll probably start looking into SmartPort hard disk emulation for the Apple II. I’m not convinced it’s worthwhile to pursue rare or non-standard floppy formats now, some of which like Kryoflux might not even be possible on the Floppy Emu hardware. Maybe I’ll look into it more once all the common disks are covered.
I read a little about the RWTS18 format, and by itself I don’t understand why it can’t be encoded in a .NIB file. RWTS18 apparently uses a non-standard sector header and checksum, non-standard macro-sectors, and non-standard encoding to get 18 sectors per track. So if you want to actually make sense of the data or reverse-engineer the game, you would need to understand all that non-standardness. But if you simply want to duplicate the disk, it seems it should be enough just to duplicate the raw disk nibbles in a .NIB file. My hunch is that RWTS18 games also include extra copy-protection checks, that can’t easily be patched without understanding the RWTS18 format, and it’s those checks that actually prevent a straight NIB copy from working.
If anyone has a disk image of Prince of Persia or another RWTS18 game in its original non-cracked version, I can take a quick look at it.
@matt thanks for the info on 2MG files. I now have 140K DOS 3.3 and ProDOS order disk images working, either naked or stored inside a 2MG file. I tried a dozen random disk images and they all worked. So I still need to tackle .NIB and also make the disk images writable, then the firmware should be good to go.
No problemo!
Some more info on NIB
“A nib image is enough information to give a person skilled in the deprotection arts the ability to make a normalized dsk out of it, if the nib actually houses copy protection to begin with. But one of the most important uses of a nib is to communicate the disk’s volume number. That lives in in a sector header – outside of the normal data areas of the disk. Some games used the volume number to tell which disk was inserted at the time, for example. So it doesn’t actually help to switch a .nib to a .dsk – unless you initialize the disk with that different volume number first.”
There’s some info here: http://www.applefritter.com/content/saving-nib-images-back-physical-diskettes (David Schmidt of ADTPro)
More amazingness – thank you Steve, I’m looking forward to evaluating this on my Apple II+ when the firmware is road-ready!
Great job!
Hi!
in this picture http://www.bigmessowires.com/wp-content/uploads/2015/05/20150507_111003_resized-copy.jpg the floppy flat cable is directly connected to the disk][ interface. Did I miss something, this is possible ? Please confirm !
TIA, Rob
Yes, it’s possible. Quoting from another post:
The Floppy Emu board can be connected directly to the 19-pin floppy connector on the Apple IIgs or Apple IIc. For the Apple II, II+, and IIe, the Emu board should be connected to your disk controller card: either the Disk 5.25 Controller Card with 19-pin connector, or the classic Disk II Controller Card with two 20-pin connectors. If using the Disk II Controller Card, be careful to orient the cable correctly, since the card’s connector is not keyed and it’s easy to accidentally connect the cable offset or backwards. The red stripe on the cable should go to the pin marked ‘1’ on the Disk II Controller Card.
Speaking about formats, currently it’s kinda messy: MAME has a flux-based format called MFI, but this is at a lower level than and therefore completely blind to things like sector organization. Something a bit more common recently is EDD format, which is produced by a utility called I’m fEDD Up! (from Brutal Deluxe Software) and an EDD board in an Apple II; this is basically the data as it comes off the Disk II, but before it’s processed by the Disk II state machine on the Disk II card. MAME finally has support for this format, but there are some issues with certain software which seems related to the emulation in MAME.