I’m sort of at a crossroads now, with the basic hardware working, and I need to decide what to do next. There’s still some remaining hardware work, of course: the keyboard and real-time clock, but neither of those is essential. Then there are possible hardware extras, like video. And there’s the entire operating system and software library to consider. My eventual goal is to support multiple programs running at once, managed by a multi-tasking OS.

I find myself gravitating towards a rewrite of the BMOW instruction set. Since the machine is microcoded, there’s nothing specifically tying the hardware to the choice and syntax of instructions. The present instruction set is nearly identical to the 6502 instruction set, which seems like a reasonable fit, but I’m getting more and more annoyed with it. I’ve been looking at the 68000 instruction set, and while its hardware assumptions (e.g. 32-bit registers) don’t match BMOW very well, it feels like a good foundation for a cleaner, simpler instruction set.

My main gripes with the current 6502-style instruction set are:

• Every combination of operation and operand is a unique instruction, which feels clunky and difficult to follow. Loading the X register is LDX, storing Y is STY, transferring A to Y is TAY. Compare this with a 68000-style MOVE SRC, DST. So the previous examples would like something like MOVE (A0), X, MOVE Y, (A0), MOVE A, Y.
• Some obvious-seeming instructions are just missing. You can push A on the stack with PHA, but there’s no PHX or PHY. There’s no instruction to add two registers: only to add a register to a value in memory.
• There are no stack-relative addressing modes. Support for the stack pointer in general is pretty minimal. This makes sense for the 6502’s single-byte stack pointer, but BMOW has a full 24-bit pointer for the stack.
• There’s no concept of an address stored in a register. The ability to auto-increment and decrement address registers is similarly hidden from the assembly programmer.
• All the opcodes are three letters. This is a small thing, but it’s annoying and makes the assembly less readable. It’s not like there’s some budget for opcode name lengths.

I’m thinking of rewriting BMOW’s microcode to implement a new instruction set much closer to the 68000’s. To the assembly programmer, BMOW would appear as three 8-bit data registers (the current A, X, and Y) and two 24-bit address registers (the AR and SP, which presently are visible only at the microcode level). The 8-bit T register would remain hidden. I think I still need it as a temporary location for implementing certain instructions in microcode, but if not, I could expose it as a fourth data register.

Exposing the address registers is the most powerful change. Consider a hypothetical program to copy 100 bytes between two memory locations. First the 6502-style:

Then the 68000-style program:

It may not be obvious, but I could implement the 68000-style instructions to run this program far faster. For instance, that MOVE (A0)+, (A1)+ could be done in 3 clock cycles, compared with 10-12 for the LDA/STA pair, because the address registers are already set up and no address arithmetic is needed. Furthermore, the 68000-style program could be a subroutine in which the actual source and destination addresses are computed at runtime, while the 6502-style program forces the addresses to be known at assembly time, or to use even slower indirect, indexed addressing.

There are some downsides to replacing the instruction set with a new 68000-style one:

• Performance may be worse in some cases with a 68000-style instruction set. In general, such an instruction set would have slightly simpler instructions, and so would require more instructions to accomplish the same work as a 6502 instruction set. For example, the 6502 instruction LDA $4000 would be translated as the two 68000 instructions MOVE $4000, A0 and MOVE (A0), D0.
• The hardware can’t easily support indexed address modes like MOVE 4(A0), D0. That instruction is supposed to load a value from a location in memory 4 bytes beyond where A0 points. That means adding 4 to A0, but there’s no place to store the result of that addition other than back into A0 or A1. So the microcode would need to save and restore the old value of A0 (slow), or make indexed address modes actually change the address register (an unwanted change in semantics), or drop support for that address mode entirely.
• All software and tools would need to be written from scratch. Currently I’m using a modified 6502 assembler, and I can sort of run existing 6502 programs with a little work. But a BMOW-68000 would be different enough from a real 68000 that I’d need to write my own assembler, and couldn’t hope to run existing 68000 programs.

One other option would be to create an instruction set modeled on the 65816, the 16-bit successor to the 6502. It does add stack-relative addressing, as well as some of the obvious-but-missing instructions from the 6502. It still strikes me as rather clunky and non-symmetric, but I should probably take a deeper look at it before jumping to conclusions.