Swift on Mac OS 9

It’s April 1, and that means it’s both April Fools’ Day and the anniversary of the founding of Apple Inc. While this year is a sober one due to current events , I think a lot of people still appreciate what people are creating and sharing to keep spirits up, whether that be music or art or…impractical programming projects. And while pranks on April Fools’ seem less and less fun, obvious jokes and whimsy, not at anyone’s expense, are still something I believe in…and even better if they actually work.

Last year I implemented the world’s best code visualizer . This year I decided to seriously attempt something that I’d thought about in the past: getting a Swift program to run on Mac OS 9.

What’s a Mac OS 9?

Twenty (!) years ago, before the macOSwe know today, there was another operating system known as “Mac OS”.It was one of the first OSs to use a GUI at all, something that we pretty much take for granted these days. It also dates from the days when only one program could run at a time; because of that, even the latest version uses cooperative multitasking to run multiple programs—that is, a program has to yield its time to let others run.If a program crashed or overwrote memory it wasn’t supposed to, there was a good chance you’d have to restart the whole system.

Mac OS 9 ran on PowerPC processors, which were also used in the GameCube, PS3, and Xbox 360; earlier versions of the OS had started on Motorola’s 68k CPU series. Its successor Mac OS Xalso ran on PowerPC when it first launched; it wasn’t until 10.4 that Apple began to switch to Intel processors instead, and 10.6 when PowerPC was finally dropped.

Mac OS X was a huge step forward from Mac OS 9 in a number of ways, including preemptive multitasking so that you could actually run multiple things at once. But Apple didn’t want to just leave OS 9 programs behind, so they did two things:

• The Classic environment set up a sandbox that looked enough like Mac OS 9 to run Classic Mac OS programs directly in Mac OS X. Because the Classic environment was itself an app, all the programs that ran inside it were protected from interfering with other Mac OS X programs and vice versa. It really was quite effective, and actually survived longer than booting into Mac OS 9 (which never received support for newer PowerPC processors). But its life ended with the switch to Intel-powered Macs—Classic was built on running the instructions in the original apps directly, only having to provide compatibility shims for libraries. Think of it like Wine / CrossOver rather than VirtualBox / Parallels .

• Carbon was a packaged-up version of the old Mac OS Toolbox APIs so that you could write Mac OS X apps the same way you always had. You basically just recompiled your app and added an extra annotation saying you were “Carbonized”. ( Sound familiar? ) Fun aside: I’ve always assumed this is why Core Foundation exists—to provide a common interface between Carbon and Cocoa. I don’t actually have a citation for this, though.

Classic ended with the switch to Intel processors back in the 2000s, but Carbon worked all the way up to last year, macOS Mojave. Apple never released a 64-bit version of Carbon, presumably to encourage developers to move to Cocoa, and with last year’s macOS Catalina, support for 32-bit apps was dropped

entirely

with very few exceptions.

What’s the goal?

Since I learned to program on Classic Mac OS, and years later spent a good chunk of my career working on Swift , I’ve had the tantalizing thought that I’d like to write a program in Swift and run it on Mac OS 9. That is,

• I write Swift source code that calls Carbon / Toolbox APIs.
• I compile it for PowerPC with (a version of) the Swift compiler.
• I package it up as necessary for Mac OS 9.
• Profit!

Is this useful? No! Absolutely not! But neither wasROSE-8, and yet I still learned a lot doing it.

As you probably guessed, I managed to accomplish this, or I wouldn’t be writing this blog post. So, without further ado, here’s a picture of a Swift Toolbox app running on Mac OS 9.2, on my friend Nadine’s Power Mac G4. (Check out that blazing fast 400MHz processor!)

I assume a good number of people reading this would like to know how to do it too!

• If you want to build your own PPC-capable Swift compiler, check out the following repositories:

  git clone https://belkadan.com/source/ppc-swift-project
  cd ppc-swift-project
  git clone -b ppc-swift https://belkadan.com/source/swift
  git clone -b ppc-swift https://belkadan.com/source/llvm-project
  git clone https://github.com/apple/swift-cmark cmark
  make  # quick start to build swiftc and the stripped-down stdlib

  Note the directory and branch names of the sub-repos, and note that they should be nested inside the ppc-swift-project repo. You will also need the mpw emulator and a copy of the Macintosh Programmer’s Workshop tools to build an actual app using modern macOS.

• If you want a prebuilt PPC-capable Swift toolchain, here’s one: ppc-swift-toolchain . Note that while I’ve put a built Swift.o in this toolchain, you’ll probably only have success with optimized code that doesn’t actually have any remaining links to the stdlib (i.e. everything is inlined away). You will also need the mpw emulator and a copy of the Macintosh Programmer’s Workshop tools.

  You may still want to check out the example in theppc-swift-project repo. The required flags for swiftc , PPCLink , and Rez can be a little finicky. (And note that the SIZE and carb resources are required for any Carbon app, so you can’t just skip the Rez part if you actually want to run your app.)

• If you just want to try a built version of BitPaint, here’s one:BitPaint-Swift.hqx.

  

I’d like to hear about anything you make with these tools! Meanwhile, if you’d like to hear how I made this work, read on.

Gathering materials

The last time I was building Classic Mac OS apps, I was using CodeWarrior . Actually, calling that “building Classic Mac OS apps” was a stretch; I was learning C and using CodeWarrior’s terminal I/O library to get a stdin/stdout interface that Classic Mac OS didn’t have natively. (Remember, no command line!) I could try to get some version of CodeWarrior running again, but that didn’t seem like the most convenient thing. I didn’t think I’d be able to get the Swift compiler running on Classic, so I’d be shuttling object files back and forth between OSs to get anything done.

Fortunately for me, I’m not the only one interested in building Classic apps on modern macOS. At some point I found about the mpw project: an emulator specifically for running Apple’s Macintosh Programmer’s Workshop tools. And I knew it was going to work, too, because Steve Troughton-Smith, (in)famous in the Apple community for finding undocumented and prerelease features in Apple’s OSs, had written up his experiences building an app with mpw that ran on System 1 all the way up to modern Mac OS X, just by building with the appropriate compiler and against the appropriate libraries.

If you’re interested in all this, I highly recommend checking out his blog post . Not only was this the reference I used to get started, but the app you see running in the above picture, BitPaint, is Troughton-Smith’s test app, ported to Swift. (I did ask him ahead of time if it was okay to use his app for a hobbyist project.) Longtime Mac developer Gwynne Raskind also gave a two-part high-level tour of the Toolbox APIs on Mike Ash’s blog several years ago ( part 1 | part 2 ); fortunately, Carbon takes care of a fair amount for us even on Mac OS 9.

So okay. What does MPW give us?

• A PowerPC compiler
• A PowerPC assembler
• A PowerPC linker
• The Classic Mac OS header files
• The Classic Mac OS library stubs, for linking against
• A bunch of object and binary inspection tools, which we don’t need for the finished product but which I made a lot of use of when trying todebug mystery misbehavior

That’s pretty good; as Troughton-Smith’s blog post shows, it’s enough to build an entire app that’ll run on Classic. My idea was to take object files produced by a modern compiler and feed them to the PowerPC linker, which means I’ll additionally need:

• A modified version of the Swift compiler that supports emitting MPW-compatible object files
• Some stripped-down form of the Swift standard library and runtime (enough to read in and interact with Carbon headers, at least)
• An actual machine running Mac OS 9. I have one, but not the charger for it, and so I did most of my testing using SheepShaver . My friend Nadine provided some testing on actual machines once things were working.

And, well, that should be it! So, off we go.

Modern compiler, classic linker

To make things more manageable, I set an intermediate goal: build an app using Clang, the modern C compiler that ships with Xcode. Clang uses the same LLVM infrastructure as the Swift compiler, so I figured I could deal with all the object format and workflow issues in Clang, and then move on to the Swift-specific parts.

The first thing I did was try to figure out what the file format was for PowerPC object files. It turns out it’s a format called XCOFF; searching for modern documentation on this turned up an IBM reference doc . Pretty much no one else uses this format, which was not encouraging. The first time I started looking into this project, I was worried I’d have to have my compiler write out assembly code and then send that through the MPW PowerPC assembler…after fixing it up to account for the differences in how LLVM and MPW print PowerPC assembly.

However, when I checked to see if LLVM supported XCOFF, I was in for a stroke of luck. It turns out IBM has started adding support for XCOFF to LLVM just last year, as part of adding support for their AIX OS … which runs on PowerPC. So I could ask Clang to generate XCOFF files for AIX, which means it should only be a short step to making it generate XCOFF files for Classic Mac OS.

At this point I remembered a bit of trivia. Apple and IBM used to have a close partnership, along with Motorola. They’d even made some common standards that were used across platforms and CPUs, though perhaps with less impact than they’d hoped. Was it possible that AIX and Classic Mac OS used the same calling conventions for their procedures, and they could just interoperate without any extra work?

I got lucky: the answer is (nearly) yes. The AIX register conventions and stack conventions match up with the ones in the Mac OS Runtime Architectures guide. That meant I could feed object files produced by Clang directly into MPW’s PPCLink and get a working Classic Mac OS binary out.

I’m pretty sure my mouth fell open when I first saw this work.

% clang -c test.c \
    -target powerpc-ibm-aix-xcoff \
    -isystem ${MPW}/Interfaces/CIncludes \
    -integrated-as \
    -fpascal-strings
% mpw PPCLink test.o ${PPC_LIBRARIES} -o Test

That should work with just top-of-master-branch LLVM/Clang (and a very simple test.c). I did end up needing to change LLVM in a few ways in the end, but it’s really fairly minimal, so much thanks to the IBM folks for doing the hard part of the work for me!

Now do Swift

Being able to compile a simple test program was a great milestone, but I had to do a fair bit of work before I could get swiftc to compile a whole BitPaint. Here are some of the highlights:

• Teach Swift about the PPC/AIX target.This mostly involved adding ppc and AIX cases to switch statements across the Swift compiler, but also involved making a simple description of the Swift calling conventions for Clang (which I cribbed from the 32-bit ARM implementation) and then assuring the PPC/AIX backend that this was an okay calling convention to be using. I got lucky in the amount of work I had to do here because Swift already supports 32-bit ARM, little-endian 64-bit PowerPC (when running Linux), and big-endian 64-bit s390x (another IBM architecture); all the pieces were already in place.

• Add support for Pascal strings.The Mac’s first high-level programming language was Pascal, not C! As such, the default format for strings throughout the Toolbox APIs was Pascal strings (a length byte followed by string data) rather than C strings (string data followed by a null byte). With the -fpascal-strings command-line flag, Clang supports static Pascal strings with the syntax "\pHello World" . The \p would be replaced by the length of the string (which must be no more than 255 bytes) so that you didn’t have to count it yourself. I hacked this into Swift as well , and while my implementation probably has problems, it was enough to get simple things working.

• Turn off reflection support and nearly all runtime metadata.The Swift runtime is very powerful, but I didn’t want to write much of a runtime for this project, which was primarily about calling a bunch of C functions. Beyond that, though, the default format for Swift metadata makes heavy use of relative addressing (mainly to reduce startup time, but learn more here ) as well as symbols pointing inside of a global, and the LLVM XCOFF implementation doesn’t (yet?) support either. So to get to a working proof-of-concept, I aggressively commented out parts of IRGen that made use of either feature. I’d like to get some of the static metadata back at some point, but reflection’s not something I’m ever interested in. Probably.

• Make a smol stdlib.The full Swift standard library has a lot of things in it I don’t need, and some that I wouldn’t even know how to implement. (What’s a String in a world that can’t assume Unicode?) But all the logic to integrate with C code is based on having some basic types in the standard library (like Int16 and UnsafeMutablePointer). What I ended up doing was taking a subset of the standard library sources, and then adding additional files and commenting things out until it worked.

  …haha, nope, even that wasn’t good enough. My early attempts at this compiled okay, but they managed to crash PPCLink when I tried to write a test program, presumably because there’s just too many symbols in the standard library. So I cut things down to an even smaller subset, and that (eventually) worked. Of course, I was working on this at the same time as I was modifying the compiler to get to a working proof-of-concept, so I think I ultimately went further than I needed to. (A bunch of symbols are only used for runtime metadata purposes.) As mentioned above, non-optimized builds of non-trivial programs don’t work yet, so I don’t know if I’m in the danger zone or not, but I might try to add a few more things back in.

  Rather than modify the actual Swift repo for this, I decided to keep my stripped-down standard library separate, so you can find it in theppc-swift-project repo. This might be a good reference for someone looking for a C-compatible, runtime-less subset of Swift , perhaps for an embedded or other resource-constrained environment.

• Disable jump tables.LLVM optimizes switch statements into jump tables when it looks like it’ll help performance and/or code size, but its default implementations of jump tables also weren’t supported in the LLVM XCOFF implementation. I imagine the AIX folks will get around to implementing this sooner or later, but for now I just disabled jump tables entirely, forcing the compiler to emit switches as a series of if s instead.

You can check out all the changes in theswift andllvm-project repos, if you’re curious. Very few are appropriate for upstreaming to their respective projects, but I’ll try to get the ones that are relevant upstreamed at some point.

A week of mysterious failures

Having made all the changes above, I had an app that worked! In Swift!

…except, it only worked some of the time. I’d change something arbitrary and suddenly events wouldn’t register any more. It got so bad that I added a counter: after any ten events, exit the app. Without that, I’d get trapped, unable to even quit without restarting the (virtual) machine. Even in a seemingly working version, my friend Nadine reported that trying to use the Reset command caused the app to crash. What was going on?

I decided I had to get to the bottom of something strange I’d seen earlier: even the Clang version of the program didn’t work correctly when I turned on optimizations. It’s possible that that was a bug in IBM’s newly-added AIX support, or 32-bit PowerPC support since it’s not such a common platform, or even LLVM’s optimizations. It could be that AIX and Classic Mac OS really weren’t as similar as I thought they were, and so my code wasn’t agreeing with the system code on how things were supposed to work. And it could be that the optimized code was using an instruction that SheepShaver didn’t support, though that didn’t really seem to match the symptoms.

And the symptoms were weird. Some local variables were getting corrupted, but others weren’t. So I started testing everything I could think of:

• was the stack aligned properly?
• was the stack pointer somehow not getting restored properly?
• was the glue code for cross-library calls trashing other data?(see the Mac OS Runtime Architectures guide)
• was there something causing the Code Fragment Manager (dynamic linker) to put the wrong address in for cross-library calls?

Without being able to rely on logging, I made the simplest textual debug output facility I could: modifying the title of a menu. I wrote C functions that tracked the current stack pointer to make sure it was getting restored properly; I made good use of the DumpXCOFF and DumpPEF tools that came with MPW; I learned how PEF “pidata” (“pattern-initialized data”) worked and tried to step through CFM relocations by hand (again, see the Mac OS Runtime Architectures guide). I even started trying to decompile some of the actual system libraries to see if they were doing anything suspicious, even though a bug in the actual Mac OS 9 seemed incredibly unlikely. This led all the way to learning about the “toolbox ROM”, which isn’t actually ROM at all: it’s a boot script and a compressed set of system libraries. (It’s called that because it’s content that used to be in ROM.) Fortunately SheepShaver already knows how to load it , which meant that I could do the same decompression and then manually split out the individual libraries.

Yeah, I got way off in the weeds. I learned a lot, though!

Finally, I looked at the decompiled optimized code—the C version, not the Swift version. I observed that the variable getting corrupted was in general-purpose register 13. That’s supposed to be an okay place to put data in Classic Mac OS (and in 32-bit AIX, and in 32-bit Mac OS X), but I decided I didn’t trust that, particularly because that register had been used to track thread-local storage in 64-bit AIX. So I marked r13 as reserved…

…and the problems went away. Optimized, non-optimized, even with -fstack-protector-all on. And Swift.

(Debugging this took about a week, unfortunately, which led to this project being a little less ambitious than I originally wanted.)

“Future directions”

What didn’t I get to? An awful lot, actually.

• There’s no runtime at all, which means no dynamic allocation (among other things).
• There’s no type metadata, which means no generics (that aren’t optimized away).
• There’s no field metadata, which means no key paths (that aren’t optimized away).
• There’s no Unicode support, so no Strings. Arguably I could make a String without Characters, or a String using MacRoman as the native encoding, but it wouldn’t necessarily look much like today’s Swift.String.
• There are a bunch of other standard library things missing because I wanted to get the proof-of-concept working, but also because PPCLink was choking on large object files. If I do get more standard library stuff working, I’ll probably split it out of the ‘Swift’ module somehow.
• I had to mess with linkage in a number of ways to make the LLVM XCOFF backend happy, so I’m not sure multi-file builds would work. I didn’t even test it.
• I’m using Carbon, which means that my program ought to work on older versions of Mac OS X as well, but my friend Nadine tried and it didn’t, and it wasn’t a priority to figure out.
• I wanted to try making nice abstractions on top of the some of the Toolbox APIs.
• I wanted to make more complicated example apps!

Maybe I’ll follow up on some of these, but I’ve been putting a lot of effort into making sure I could finish this by April 1, so I should probably get to some of the things I’ve been neglecting in favor of this project instead.

Summary

This project took a lot of time, even though I (1) know a lot about compilers and (2) hacked my way to success instead of being careful and maintaining proper software development practices. But I learned a lot, and I accomplished a goal I’ve had in the back of my mind for a long time.

If you made it all the way to the end of the article, here’s a reward: BitPaint running under Classic on Mac OS X 10.2 (also courtesy of Nadine).

Stay safe, everyone, and help the people around you when you can. And if anybody makes something with this project, I want to hear about it!