Ever since Doom was made open source, porting the game to run on anything with a display has been a hobby for programmers. From digital cameras to web browsers, the game has touched any computer capable of running the code.

Last May, NCommander released a YouTube video documenting the perils of porting Doom to run on an IBM RS/6000-series computer from the early 2000s, which cost at least US$10,000 at the time of its launch and ran AIX, a UNIX operating system by the same manufacturer.

I’ve never considered porting Doom before, but I was curious if my favourite abandoned UNIX variant had a port. Some very brief cursory searches on Google and GitHub, I was led to believe that this was unlikely or if it had been done, it was never publicly announced a port or it has been lost to the sands of time.

If it is the case nobody has bothered, there is a good reason: it isn’t exactly necessary. Once I explain A/UX, it’ll make sense why I am likely the first person ever to port the game to this platform.

What is A/UX?

Prior to Mac OS X (now macOS), Apple did have another operating system built on the foundations of UNIX. A/UX was Apple’s UNIX implementation and it is a true UNIX in that it ran AT&T System V under the hood, just like IBM AIX.

Its purpose was to give Apple a foothold within the United States government’s computer contracts as it provided the ease of the Finder interface with the power of UNIX, which was the backbone of many projects.

And all of this sounds weird already I am sure you are thinking; it is. A/UX is UNIX but it runs the Macintosh System software as its own thread. This meant that your copy of Microsoft Office or Adobe Photoshop could run simultaneously with something like a COBOL application initially written in the 1970s on the same computer.

Apple maintained the operating system from 1988 through to 1995, at which point it was abandoned as it was written solely for the Motorola 68000 series computer models — the company had moved on to PowerPC.

Hardware requirements for the operating system were rather steep. Most entry-level machines could not run it (really the vast majority including all PowerBooks) and the base models of any qualifying computer were often not sufficient either. It wasn’t ever meant to be used by the general public.

Other projects like MkLinux and Copland came along around its demise as well, but they too were abandoned when Next was bought by Apple and subsequent development on Rhapsody led to the release of Mac OS X, ending the use of the legacy operating system Apple was formerly chained to.

Interestingly, both Mac OS X and A/UX share one thing in common: the ability to execute classic Mac applications atop of a multithreaded operating system. Macintosh System didn’t have any true multitasking and was a rather inferior operating system in contrast, but these two operating systems got around that problem in a rather effective manner.

Preparing to port Doom

I have had a Quadra 800 with 64 MB of RAM sitting in my closet for a long time. I’ve streamed it a few times and have it to thank for a speedrun that failed due to a softlock (Mario’s Time Machine Deluxe, which I hold the solo world record for on the Windows release), but it has never really been used to its full potential.

A/UX has been something I have always been curious about and I’ve maintained an SD card with an install of it for my Quadra for a few years — this is helped by my SCSI2SD. Having seen Doom ported to AIX with all of its challenges made me wonder about doing it myself.

I myself am not very confident in my programming abilities despite my online username, but I figured I was capable enough so I went forward. My primary language is Python, but I had written C on embedded systems and figured my knowledge was enough to take a crack at this.

The latest and final release of A/UX was version 3.1.1 in 1995. Doom was released a few years prior and was itself developed on NeXTSTEP, meaning that the code should with some work port over quite easily as it was meant to use X11 from the ground up — this is the windowing system the majority of UNIX operating systems used then.

June 8, 2022 — Whoa. What a mistake I made with the last paragraph. NeXTSTEP used Display PostScript, which isn’t like X11 at all. I completely overlooked this fact considering OS X’s display mechanism. Anyway, I am leaving the above as is.

A/UX has an X11 environment, but I wanted to use its full experience: the Finder environment it came with — you can switch between them easily at the login prompt.

You have a choice between a console emulator, a 24-bit or 32-bit Finder, or X11.

In order to achieve this, this required the use of MacX, an X11 implementation also abandoned by Apple. This software permits the use of the system on any Mac seamlessly within the Finder — and it is not just limited to A/UX-running systems either, but that is a different topic all together.

“High-performance” by 1997 standards.

Having the base system ready, I also needed some other tools: namely the GNU GCC compiler and its various components.

Fortunately, someone who worked at NASA had for almost a decade after A/UX’s demise maintained ports of modern-ish software — a mirror is available here. This meant that I had the ability to use version 2.7.2 of GCC plus a myriad of other required tools to both compile the software and also not make me pull my hair out. Additionally, you can find other useful software including OpenSSH via other mirrors, which made transferring files easy.

As for Doom itself, the source code has been freely available for decades. All you need to make it run post-compilation is a WAD file, which contains all of the assets for the game to run. Most Doom releases over the decades have included this file in one form or another and I had both the shareware and retail releases handy.

I had the tools and the pieces, but now I had to do the hard work.

You can’t just make this stuff up

Often when you get source code, you can just browse to the directory and type ‘make’ at the command line to have it compile into something useful. This is almost the case here with Doom on A/UX, but its codebase expects you to be building for Linux instead and very quickly you are presented with problems.

Sound was designed for Linux for one and unfortunately isn’t simple or consistent across UNIX and UNIX-like operating systems — OSS is not an option either. I didn’t feel like dealing with this mess, so I commented out all instances of sound and music in the source code. I could have just changed the code to just do nothing, but I was admittedly looking for the quick way out rather than build something flagged at compile time to contend with this.

Being that A/UX is also ancient in the operating system sense, a lot of functions that would be included in anything made in the past thirty years are missing. Fortunately, these functions are limited to the use of alloca, which can be replaced by malloc (with some extra code); and strcmpi, which I just commented out.

The latter’s removal had a slight consequence: the WAD file, which contains the game content, now must have a lowercase filename as that function’s task is to do a string comparison without case sensitivity. It didn’t exist in GCC until just after the turn of the millennium so we either had to remove it or write something in its place.

Doom does a file scan in the same directory as where it is executed from for files matching the extension “wad” upon launch, so my workaround is an annoyance but reasonable.

One joke about the Motorola 68000 architecture is that it’s basically “backwards x86” — 68000 is abbreviated as “68k” and if you reverse it you get “k86” (get it?). At an instruction level, there is an unsurprising overlap between the two, but honestly where the joke lands at is that it’s big endian as opposed to little.

This presents a problem with these WAD files as their byte order is meant for a little endian system, which while is the case for x86, it is not for the 68000. I had to add compiler flags to make use of the built-ins which manage this problem and fortunately that was the last hurdle to get the game to actually load and seemingly function on its own.

Reversing a suggestion to run these as signed as opposed to unsigned was the last hurdle after adding a compiler flag to force use of the functions if the flag is used. After this point, I had success!

One small thing: I had left networking untouched in the code, but I disabled at compile time as it was presenting problems. However, I think it would be neat to take NCommander’s suggestion of trying to do something akin to a deathmatch between two different dinosaur systems.

It works, but it’s slow and has other problems

After two evenings worth of streaming on Twitch my making code changes and testing on actual hardware, one compile finally led to it executing. I actually didn’t expect the last fix to be the final task remaining to make it run as expected, but it did and for the first time we had Doom running natively on A/UX in an X11 window within Finder.

Aside from the lack of sound and having to tell MacX to let me use the arrow keys as desired (apparently mouse emulation with them is a default), the game was running at maybe 2–3 frames per second. Lowering the graphic details improved things a bit, but adjusting the overall screen size in-game netted no change.

Becky Heineman’s README file for the 3DO Doom port comes to mind now.

Keybindings also restricted me to anything but move around, change weapons, select menu options, and open doors — I was unable to shoot enemies, which is an objective in winning the game. However, I was able to get from the start of the first level in Episode 1 to the exit, meaning that I was able to play the game in a pacifist setting.

I made it to the end of the level with no kills, but I did at least one secret! Also, someone asked if this was real hardware, so I made my OSSC show the display details.

But the frame rate problem remained. The running theory I have is that since MacX is running the game in Finder which itself is a process in A/UX, the game has to traverse through the TCP network stack to output to Finder, meaning there is additional CPU overhead. The 33 MHz processor is handling the operating system, Finder, MacX, networking, and Doom all at the same time, so as a consequence, the visuals suffer.

The first thing I did to tackle the problem was to set the compiler flags to automatically optimise the code just so we can at least improve the existing situation. This is all handled by GCC itself with a simple flag and the consequences of doing so should be (and “should be” is key) additional memory use and longer compilation time.

It didn’t add much to the latter and the memory use was inconsequential, but the graphics rendering absolutely suffered with little improvement to the frame rate. Effectively I turned a slideshow that looked otherwise normal into a slideshow resembling a music video for a vaporwave single. So let’s not enable this option for now I guess.

At least the HUD was normal.

I tried to find a reason for why using this flag led to this problem, but it could be many things and the Makefile in the original source doesn’t have them enabled anyway.

The problem I have right now is that X11 doesn’t wish to be a solo session on my A/UX install and I cannot figure out why. My only guess is that maybe it might hate my Radius graphics accelerator, but I haven’t had the time to diagnose this.

However, I am confident that the performance problem resides with the sandwich Doom must crawl through to make it to my eyeballs and that running it in a pure X11 session is the best way to go.

Feel free to figure this out for yourself however!

Why am I (likely) the first to do this?

I think there are two reasons.

The first is that A/UX is obscure. It never really had a market outside of the US government and organisations associated with it. Even with its largest customer, it had an extremely small user base and its abandonment accelerated its irrelevance. You’re more likely to find an RS/6000 running AIX today than a Quadra 900 running A/UX.

The obscurity is also hastened by hardware it runs on often never being intended for the at home market. The only mainstream Mac I can think of capable of running it was the SE/30. It’s “mainstream” in the sense that it appeared in Seinfeld as a prop in Jerry’s apartment for several seasons.

Additionally, with it pre-installed on a Macintosh, it would set you back almost US$10,000 in the late 1980s, or about US$25,000 in 2022 money. This was never meant for the average end user!

The second reason is the least likely but came to mind: Doom sort of can run on A/UX already. When I say “sort of”, I am referring to A/UX’s Finder implementation which gives a System 7.0.1 environment. The caveat here is that Doom requires System 7.1, but with a few hacks and tweaks, you can run a later version of the system software albeit with the loss of certain components which makes A/UX itself useful alongside Mac software.

My Quadra’s processor while sluggish should not be a problem in running this game I must add, but these older CPUs are single-threaded and can only do much at a time. Doom may seem like a really simple game by modern standards (and it is really a glorified 2D one), but it was cutting edge when it came out in the 1990s.

Overall, because of its obscurity and hardware requirements plus emulation capable of running the OS still in its infancy (Shoebill and QEMU are the only options as of this writing), it is likely that my hobbyist exposure plus my love of Doom finally led to an attempt to build the game to run on the platform.

The help I received when I streamed my porting attempts definitely made this possible as well, so it was not a solo effort I must admit.

Up until this point, my knowledge of any operating system development was limited to Linux, macOS, and Windows, but I guess I can now say I know a little bit about AT&T System V and its A/UX variant.

Where can I get this?

The source code is available on my GitHub and a compiled binary is available there too. I have no intention to update this further beyond maybe a few tweaks to make it more playable. If you wish to make changes on your own, feel free to fork it or just send me pull requests.

Make sure to read the README before compiling as there are a few steps you might need to take in order to get it up and running.

I have no idea if the emulators capable of running A/UX can even run this game and I have even less of a clue whether you can run this on anything earlier than version 3.1.1 of the operating system.

I have the two streams where I worked on it archived on YouTube (June 5th and June 6th were their recordings), but do give me a follow on Twitch as well as I stream speedrunning, visual novel games, and occasionally stuff like this.