Friday Q&A 2009-01-16

Happy Friday to everyone, and welcome back to another Friday Q&A. This week I'll be taking Eren Halici's suggestion to discuss the various ways to do interprocess communication on OS X.

IPC is an interesting and sometimes complicated topic, especially on OS X, which has a veritable zoo of IPC techniques. It can be hard to decide which one to use, and sometimes hard to even know what's available.

OS X is a funny mixture of mach and UNIX so you end up with IPC mechanisms from both:

• Mach ports: The fundamental IPC mechanism in mach. Fast, light-weight, extremely capable, and difficult to use. Mach ports will not only let you talk to other processes, but do things as drastic as inject code into other people's programs. The poor state of the mach documentation makes it hard to get started and easy to make mistakes with it. On the other hand, the core mach_msg function is probably the most optimized syscall in the system, so they're really fast to use, and your machine will barely blink if you decide to allocate million mach ports at once.
  • CFMachPort: A very thin wrapper around mach ports. CFMachPort essentially exists to allow a mach port to be used as a runloop source. It can also help with creating and destroying the ports. It helps a little with receiving messages and not at all with sending them.
  • CFMessagePort: This nice CoreFoundation wrapper around some mach functionality makes it easy to set up synchronous back-and-forth communication between two unrelated processes. You can start a server with just a few lines of code. Another program can then look up that server by name and message it. You get the speed advantages of mach without all the messy stuff going on underneath.
  • NSPort/NSMachPort/NSMessagePort: Cocoa has some mach port wrappers too. They're mainly geared toward use with Distributed Objects (more on that below) but can be used on their own as well, if you're brave.
• POSIX file descriptors: There are actually several kinds of these but they can all be used with the typical read and write calls once they're set up.
  • Pipes: The archetypal POSIX IPC mechanism. If you've ever used the | pipe operator in a UNIX shell, you've used a pipe. Pipes get created in pairs within the same process, so they're good for communicating between parents and children (or between two children of a single, coordinating parent) but not so good for communicating between unrelated processes. Make them with the pipe call.
  • FIFOs: It's like a file, but it's like a pipe! A FIFO gets an entry in your filesystem, just like a file, but writes don't go to the filesystem, instead they go to whatever process has opened the fifo for reading. You can make these with the mkfifo call. The end result is a pipe that has a filesystem entry, which can make it easy for two unrelated processes to hook up. The processes don't even have to know that they're talking to a fifo. Try it out in your shell:

        $ mkfifo /tmp/fifo
        $ cat /tmp/fifo
        
        # in another shell
        cat > /tmp/fifo
        type some junk here
    

  • Sockets: You probably know these from working with TCP/IP, but they can also be used to communicate locally, and not just by connecting to localhost. If you create a socket in the AF_UNIX family you get a socket that's only for local communication and uses more flexible addressing than TCP/IP allows. AF_UNIX sockets can be created using a filesystem path much like a FIFO by using the socket and bind calls, but allowing multiple clients and more options for how the communication works. They can also be created anonymously using the socketpair call, giving you something much like a pipe, except bidirectional.
• Shared memory: Shared memory is a magical piece of memory which appears in multiple processes at once. In other words, you write to it from process A, and read from it in process B, or vice versa. This tends to be very fast, as the data itself never touches the kernel and doesn't have to be copied around. The downside is that it's really difficult to coordinate changes to the shared memory area. You essentially get all of the disadvantages of threaded programming and most of the disadvantages of multi-process programming bundled together in one neat package. Shared memory can be created using either mach or POSIX APIs.
• Miscellaneous, not really IPC: There are some techniques which don't really count as "IPC" but can be used to communicate between programs if you want to.
  • ptrace: This system call exists mainly for writing debuggers, but could in theory be used to do non-debugger things too. Not recommended, included only for completeness.
  • Files: Sometimes it can be useful to communicate using plain old files. This can be as simple as creating a lock file (a plain empty file that works simply by being there) for mutual exclusion, or you can transfer actual data around by writing it to a file, then having the other program read it. This tends to be inefficient since you're actually writing to the filesystem, but it's also easy and nearly universal; every application can read files!

Those are all what I would call system-level functionality, things which are either provided directly by the kernel/libSystem, or which are thin wrappers around them. OS X also provides a bunch of higher-level IPC mechanisms at the framework level:

• Apple Events: Scourge of the Skies, Champion of the Ugly Contest, King Slow, Emperor Horrible. Apple Events are all of these things, but they're also tremendously useful. They're the only IPC mechanism which is universally supported by GUI applications on Mac OS X for remote control. Want to tell another application to open a file? Time for Apple Events. Want to tell another application to quit gracefully? Apple Events time. Underneath it all, Apple Events are built on mach ports but this is mostly not exposed in the API.
  • AppleScript: Everything Apple Events is and worse, but still often useful, AppleScript is a scripting language built on top of Apple Events. Generally it's best to avoid AppleScript and simply send the corresponding raw Apple Events instead, either directly or through a mechanism like Scripting Bridge. AppleScript support is the standard way to allow users to script your application, although if you ever try to add AppleScript support to your application you'll find yourself wishing for a different standard.
• Distributed Objects: It's like Objective-C, but it happens over there! DO gives you proxy objects that can be used (mostly) just like local objects, with the exact same syntax and everything, except that your messages fly across to the other process and get executed there. DO normally runs over mach ports but can also be used with sockets, allowing it to work between computers as well. DO is really cool technology and it's the sort of thing that tends to blow people's minds when they come to Objective-C from lesser languages such as Java or C++. Unfortunately DO is also really old and crufty and tends to be strangely unreliable. This is especially true when using it with sockets to talk to remote machines, but is even true when using it locally. DO is also completely non-modular, making it essentially impossible to swap out the IPC mechanism it uses for something custom (like if you want to encrypt the stream). It is worthy of investigation if only to learn about how it works, and despite the shortcomings can still be very useful in certain situations.
• Distributed Notifications: These are simple one-way messages that essentially get broadcast out to any process in the session that's listening for them. Extremely easy to use, and available in both Cocoa and CoreFoundation flavors. (And they interoperate!) The downside is that they don't guarantee delivery and they're very resource-intensive due to potentially messaging every application on your system. They would be completely unsuitable for something like transmitting a large picture to another process, but are great for simple one-off things like "I just changed my preferences, re-read them now". Internally this is implemented by using mach ports to talk to a centralized notification server which manages the task of getting notifications to where they want to go.
• Pasteboard: Probably the IPC mechanism that you've directly used the most. Every time you copy and paste something between applications, that's IPC happening! Inter-app drag and drop also uses the pasteboard, and it's possible to create custom pasteboards for passing data back and forth between applications. Like distributed notifications, pasteboards work by talking to a central pasteboard server using mach ports.

That wraps things up for this week's Friday Q&A. Come back next week for another exciting installment. Did I miss your favorite IPC mechanism? Don't understand the point of one of them? Did I miss the whole point of one of them? Post your comments below.

Remember, Friday Q&A is driven by your suggestions. I'm much easier than Public Radio: I don't ask for your money, just your ideas, but your contributions make this possible. Post your ideas in the comments, or e-mail them. (Reminder: I will use your name unless you tell me not to.)

Comments:

Add your thoughts, post a comment:

Spam and off-topic posts will be deleted without notice. Culprits may be publicly humiliated at my sole discretion.