Collapsing Futures in Objective-C

posted by Craig Gidney on October 8, 2013

In this post: practical examples of benefits of using collapsing futures in Objective C (library available on GitHub).

Experimenting

Returning readers may remember I’ve posted in the past about futures. I wondered if separating may-fail-ness from eventual-ness might be fruitful, and if making futures automatically unwrap when nested would be a good idea (in weakly typed languages where that didn’t violate the type system).

I’ve had the chance to try these ideas out, in Objective-C. I’ve learned two things:

Separating may-fail-ness from eventual-ness is a terrible idea. The number of times you don’t care about failure is somewhere in the neighborhood of never (mainly because it’s so useful to be able to cancel asynchronous operations, and that means a failure case). Even worse, the code tends to end up as a mess of unwrap-unwrap-advance-wrap-wrap no one can keep straight.
Collapsing futures are amazing.

Since collapsing futures worked out, I’ve extracted my implementation into a tiny library for Objective-C. It contains just two types: Future and FutureSource. For the rest of this post I’ll be giving practical examples of using them, demonstrating how they simplify lots of tasks.

Update: The library now also contains cancel tokens, and will likely grow more in the future. Types are also now prefixed with TOC. The rest of this post has been updated to match the latest version as of Oct 15.

AddressBook as a Future

Our first practical problem is… requesting the address book in iOS.

I consider this to be an intermediate level problem, because the API for it is kind of awful. There’s lots of details you have to get right all at the same time, and it’s really easy to miss a case if you don’t read the documentation carefully and test things out.

As an example of this not being trivial, the answers to the question on StackOverflow all have flaws. The top answer fails to check that access was actually granted (an understandable oversight when giving an answer, but any bets on how often that bug has been copied now? Update: has now been fixed.). The second place answer recommends using a deprecated method and blocks on the result (Don’t block on async stuff. Ever. It makes your code brittle and prone to deadlock.).

(I was also worried that all the answers had race conditions vs the user taking away authorization. Turns out your app gets sigkilled and restarted when that happens. That’s… actually a pretty fail-safe way to prevent apps thinking they have access when they don’t.)

Our goal is to hide all of the difficulty related to using this API and expose our simple method instead. It will just eventually get the address book, requesting access if necessary and failing if access is not granted. Here’s my implementation:

+(TOCFuture*) asyncGetAddressBook {
    CFErrorRef creationError = nil;
    ABAddressBookRef addressBookRef = ABAddressBookCreateWithOptions(NULL, &creationError);
    assert((addressBookRef == nil) == (creationError != nil));
    if (creationError != nil) {
        return [TOCFuture futureWithFailure:(__bridge_transfer id)creationError];
    }

    TOCFutureSource *futureAddressBookSource = [TOCFutureSource new];
    
    id addressBook = (__bridge_transfer id)addressBookRef;
    // (assuming IOS 6 or higher)
    ABAddressBookRequestAccessWithCompletion(addressBookRef, ^(bool granted, CFErrorRef requestAccessError) {
        if (granted) {
            [futureAddressBookSource trySetResult:addressBook];
        } else {
            [futureAddressBookSource trySetFailure:(__bridge id)requestAccessError];
        }
    });
	
    return futureAddressBookSource.future;
}

Before I explain what this does, notice how stupid the workflow to get the address book looks:

We have to create the address book then ask if creating it was a useless thing to do.
The two failure cases (user already denied permission, user chooses to deny permission) arrive in two totally different contexts.
The success case is also artificially split into two success cases (succeeding inline vs succeeding later).
We’re tempted by being able to complete inline, but ultimately we can only actually rely on eventually completing.

Bleh. Anyways, on with the explanation.

We start by trying to create an address book reference by calling ABAddressBookCreateWithOptions. According to the documentation this method returns nil, and an error, only when the user has already denied us access in the past. In that case we immediately forward the error to the caller by giving it to the futureWithFailure method and returning the resulting future. (We also transfer the error into ARC, since we have to release it.) If there was no error creating the address book, we continue to the other cases.

In order to determine if user has authorized or will authorize us to access the address book, we use the ABAddressBookRequestAccessWithCompletion method. It’s an asynchronous method, meaning it calls us back later, so we create a TOCFutureSource to set at that time. We return the source’s future for our caller to use right away.

Once the user’s choice about whether or not we can have access is known, the callback we’ve specified is called. If we were granted access then we make our future succeed with the address book by giving it to trySetResult. Otherwise we were denied access, and forward the error to the caller via the future by giving the error to trySetFailure. (The created address book is not leaked because we transferred it into ARC earlier.)

Here’s how a caller could use our method:

TOCFuture* futureAddressBook = [Util asyncGetAddressBook];
[futureAddressBook catchDo:^(id error) {
    NSLog("Oh No!");
}];
[futureAddressBook thenDo:^(id addressBook) {
    // hurray!
}];

Notice how the unreliable inline cases have been folded into the reliable callback cases. Instead of having to deal with a very odd API, the caller deals with a TOCFuture that behaves just like any other TOCFuture.

Connect the Futures

The most useful method on TOCFuture is then. It lets you specify a function to run once the future’s result is ready, and returns a future for the result of that function eventually finishing. If any failure occurs along the way, it just propagates the failure instead of running callbacks. There’s also a variant, thenDo, which takes functions that don’t produce a value.

The following example is adapted from an app I’ve been working on, which needs to establish calls. This involves way more machinery than I can cover, but here’s one of the pieces which uses then several times:

/// Eventually connects to a call, resulting in a CallConnectResult containing
///    the ShortAuthString to display and the audio socket to communicate with.
/// The connection ends when the given cancel token is cancelled.
+(TOCFuture*) asyncConnectToPhoneCallDescribedBy:(Session*)session
                                  untilCancelled:(id)untilCancelledToken {
    require(session != nil);
    
    TOCFuture* futureUdpSocket = [self asyncRepeatedlyAttemptConnectToRelayDescribedBy:session
                                                                        untilCancelled:untilCancelledToken];
    
    TOCFuture* futureRtpSocket = [futureUdpSocket then:^(UdpSocket* udpSocket) {
        return [RtpSocket rtpSocketOverUdp:udpSocket];
    }];

    TOCFuture* futureZrtpHandshakeResult = [futureRtpSocket then:^(RtpSocket* rtpSocket) {
        return [ZrtpManager asyncPerformHandshakeOver:rtpSocket
                                       untilCancelled:untilCancelledToken];
    }];
    
    return [futureZrtpHandshakeResult then:^(ZrtpHandshakeResult* zrtpResult) {
        AudioSocket* audioSocket = [AudioSocket audioSocketOver:zrtpResult.secureRtpSocket];
        
        NSString* sas = zrtpResult.shortAuthenticationString;
        
        return [CallConnectResult callConnectResultWithShortAuthenticationString:sas
                                                                  andAudioSocket:audioSocket];
    }];
}

The above code is basically just what you’d write if everything was synchronous, except indented inside then callbacks. Anyways, an overview of what’s happening:

First, we need a UDP socket that has introduced itself to a relay server. The mechanics of that process are complicated, with timeouts and retries and cleaning up when cancelled and all that fun stuff. Luckily we don’t have to care: the eventual result is represented directly by the TOCFuture returned by asyncRepeatedlyAttemptConnectToRelayDescribedBy. We call the method and use its result like any other future, without a care in the world about the underlying details.

Second, we wrap that eventual UDP socket into an RTP socket. When the relay connecting stuff succeeds, and the UDP socket future gets a result, the callback that wraps the UDP socket into an RTP socket will be run. So futureRtpSocket ends up with an eventual RTP socket. On the other hand, if we fail to connect to a relay, the UDP socket future will fail and the then method will forward that failure into the RTP socket future without running the callback.

Third, we do a ZRTP handshake over the eventual RTP socket. This is another very complicated step, involving a whole sequence of packets, protocol negotiation, cryptography, resend limits, and so forth. But, again, we only have to care about the future result and we can use it like any other future. Automatic collapsing saves us a bit of trouble here, because we don’t have to think twice about the result of our continuation already being a TOCFuture. The doubly-eventual result is just automatically a singly-eventual result.

Finally, we repackage our eventual handshake result into a CallConnectResult and return. The repackaging just involves throwing away the information we don’t need and wrapping a socket for sending audio around the secured RTP socket.

What this method actually does is extremely complicated. Writing it with ad-hoc with callbacks is likely to be a world of pain. At every stage you need to deal with propagating success, propagating failure, and cancellation. With collapsing futures… the code is basically just what you’d write if it was synchronous.

Catching a Recursive Future

The second most useful method on futures is catch (and catchDo). It is the mirror image of then: instead of running when the future succeeds, it runs when the future fails.

My example for catch is a method used indirectly by the previous example. It tries to perform an operation several times, with increasing timeouts, until the operation either succeeds or fails within the time limit.

I don’t know if you’ve ever tried to implement something like that… but it’s generally awful. Especially if you want to handle success, failure, and cancellation. Collapsing futures make it look easy:

+(TOCFuture*) asyncTry:(CancellableOperationStarter)operation
            upToNTimes:(NSUInteger)maxTryCount
       withBaseTimeout:(NSTimeInterval)baseTimeout
        andRetryFactor:(NSTimeInterval)timeoutRetryFactor
        untilCancelled:(id)untilCancelledToken {
    
    require(operation != nil);
    require(maxTryCount >= 0);
    require(baseTimeout >= 0);
    require(timeoutRetryFactor >= 0);
    
    if (maxTryCount == 0) return [Future futureWithFailure:[TimeoutFailure new]];
    if ([untilCancelledToken isAlreadyCancelled]) return [Future futureWithFailure:untilCancelledToken];
    
    TOCFuture* futureResult = [AsyncUtil raceCancellableOperation:operation
                                                   againstTimeout:baseTimeout
                                                   untilCancelled:untilCancelledToken];
    
    return [futureResult catch:^(id error) {
        bool operationDidNotTimeout = ![error isKindOfClass:[TimeoutFailure class]];
        if (operationDidNotTimeout) {
            return [TOCFuture futureWithFailure:error];
        }
        
        return [self asyncTry:operation
                   upToNTimes:maxTryCount - 1
              withBaseTimeout:baseTimeout * timeoutRetryFactor
               andRetryFactor:timeoutRetryFactor
               untilCancelled:untilCancelledToken];
    }];
}

Reading from start to finish:

Make sure the input makes sense.
If you’re out of tries, fail due to a timeout.
If you’ve been cancelled, fail due to a cancellation.
Otherwise start the operation, using the current timeout.
If the operation succeeds, that’s your result.
If the operation failed, and it wasn’t due to a timeout, fail with that error.
Otherwise you should repeat this whole process but with a longer timeout and one fewer tries.

Notice that this method doesn’t do much except retry. Running a cancellable operation with a timeout, the most complicated sub-piece, is its own method. Just another example of futures not getting in the way of abstracting.

Another thing to notice is how this method exploits the auto-collapsing property. Did you see how it forwards errors into the resulting future? Instead of needing a side-channel, like raising an exception, it just returns a failed future containing the error. The future the failed future would have ended up in will collapse to just contain the error.

Also, automatic collapse is making recursing easier. If asyncTry has type TOCFuture(T), then a continuation returning asyncTry should have type TOCFuture(TOCFuture(T)). This mismatch between the type of what we’re suppose to return and what we’re returning happens anytime we asynchronously recurse, but it’s of no consequence thanks to collapsing making those two types equivalent.

Future after finally Animated

The final basic method on TOCFuture is… finally (and finallyDo). finally runs the continuation you give it when the future completes, whether or not the future succeeded or failed. It’s generally useful for scheduling after-the-fact cleanup, and for transitioning out of futures and into another async mechanism.

The following method from some demo code I used to prompt the user, using finallyDo:

-(TOCFuture*) presentViewController:(UIViewController*)view
                        untilResult:(TOCFuture*)future
                      withAnimateIn:(BOOL)animateIn
                      andAnimateOut:(BOOL)animateOut {
    
    require(view != nil);
    require(future != nil);
    
    TOCFutureSource* futureResultSource = [TOCFutureSource new];
    
    [self presentViewController:view
                       animated:animateIn
                     completion:nil];
    
    [future finallyDo:^(TOCFuture* completed) {
        // (assuming the future completes on the main thread)
        [self dismissViewControllerAnimated:animateOut
                                 completion:^{[futureResultSource trySetResult:completed];}];
    }];
    
    return futureResultSource.future;
}

This method displays a view controller (i.e. a UI thingy). It displays it until the given future completes. It returns a future that will complete with the same result as the given future, except the resulting future won’t complete before the view has animated away.

It should be clear how this method works, since it’s really the simplest example so far. It uses finallyDo to schedule dismissing the view controller. It then uses the dismissal’s completion callback to propagate the completed future’s result into our result. Note that the returned future uses the given future as its value without having to unwrap it, which is fine thanks to automatic collapsing.

I used this code in some simple demo views that pop up like modal dialogs and ask for information. The views expose a future that completes when the user accepts what they entered, or fails if they hit cancel. So asynchronously prompting for information is just a matter of making the view, passing it to this method along with its completion future, and continuing off of the resulting future.

Notes

I hope those four cases have been useful examples of collapsing futures having practical utility. The examples really just scratch the surface of what you can do. They don’t even combine multiple futures together (e.g. race multiple TCP connections, to do a low latency connection), demonstrate how it’s easier to fall into the pit of success with futures, or really drive home how well futures and cancellation tokens work together. Alas, this blog post is already well over the usual size and I do need sleep to function.

Note that, although I’ve been picking on callbacks a bit, it is possible to use them well if you’re structured and disciplined. I do dislike them, since they force you to either write your code backwards (the last callback needs to be ready before the first method in a chain), have monster indentation, or over-extract functions and lose the benefit of closures… but there’s a large component of personal preference there as well. (Also it’s harder to write general utility methods like catch for callbacks because you end up needing a variant per function signature.)

The collapsing future library I implemented is bare bones. I thought about including cancel tokens as well, but cut them because I’ve been programming in Objective-C for less than a year and I don’t feel confident publishing anything large. I know essentially nothing about what should and shouldn’t be done when designing an Objective-C library. Is it a big deal that you can’t copy a TOCFuture? In what cases can I rely on a block being copied off of the stack, instead of cargo-cult copying to be sure? Should I be prefixing TOCFuture and TOCFutureSource with something? How do I make documentation work? Should there be a single public “LibraryName.h” header that consumers will include? You get the idea.

Update: There is now a podspec included in the library, to allow installing it with CocoaPods.

Summary

Collapsing futures make asynchronous code easier. I implemented a minimal collapsing future in Objective-C library.

Discuss on Reddit

My Twitter: @CraigGidney

2 Responses to “Collapsing Futures in Objective-C”

Alex Skorulis says:

October 10, 2013 at 7:15 am

This looks like something that would really help with my current project.

CraigGidney says:

October 10, 2013 at 3:47 pm

It’s license as Simplified BSD. There’s already a License.txt file in the repo that says so. Do you mean your company wants me to put a notice at the top of every source file?

Twisted Oak Studios offers consulting and development on high-tech interactive projects. Check out our portfolio, or Give us a shout if you have anything you think some really rad engineers should help you with.

Collapsing Futures in Objective-C