Terrific Failure: Metaprogramming, code generation, ​and DI frameworks.

In Karumi over the last few months we have been trying to generate a Sourcery template that would help us to automate the composition root file we usually have to deal with all the dependencies generation ceremony, but due to Sourcery templating nature and some limitations in our design, we were unable to end up having something good enough to be confident to use in production with our clients.

We defined a clear list of requirements for that tool:

• Easy to use.
• Fast.
• Compile-time safe.
• No runtime risks.
• Zero code intrusiveness.

To sum up, we did want the same file we handcraft be automatically generated with the smallest human interaction possible, without polluting our code base with factories or builders.

Chapter I: First, do it with your hands.

In all our projects we usually have a CompositionRoot/ServiceLocator instance that helps us to provide all the instances to isolate objects usage from objects creation.

Suppose we have a bunch of classes in our project:

class Broadcast {}

class ResultDatastore {}

class UpdateStatusLocally {
    private let datastore: ResultDatastore

    init(datastore: ResultDatastore) {
        self.datastore = datastore
    }
}

class TimeProvider {}

class CMSamplesToVideoRecorder {}

class BroadcastExtensionModel {
    private let broadcast: Broadcast

    private let resultDatastore: ResultDatastore
    private let timeProvider: TimeProvider
    private let updateStatusLocally: UpdateStatusLocally
    private let videoRecorder: CMSamplesToVideoRecorder

    init(broadcast: Broadcast,
         resultDatastore: ResultDatastore,
         timeProvider: TimeProvider,
         updateStatusLocally: UpdateStatusLocally,
         videoRecorder: CMSamplesToVideoRecorder) {
        self.broadcast = broadcast
        self.resultDatastore = resultDatastore
        self.timeProvider = timeProvider
        self.updateStatusLocally = updateStatusLocally
        self.videoRecorder = videoRecorder
    }
}

To wire them up and use them, we are going to implement a class that will be responsible for building and providing instances of classes.

import Foundation

public class CompositionRoot {
    public static var shared: CompositionRoot = CompositionRoot()

    public func provideBroadcastExtensionModel(for broadcast: Broadcast) -> BroadcastExtensionModel {
        return BroadcastExtensionModel(broadcast: broadcast,
                                       resultDatastore: provideResultDatastore,
                                       timeProvider: provideTimeProvider,
                                       updateStatusLocally: provideUpdateStatusLocally,
                                       videoRecorder: provideCMSamplesToVideoRecorder)
    }

    public lazy var provideResultDatastore: ResultDatastore = {
        return ResultDatastore()
    }()

    public var provideUpdateStatusLocally: UpdateStatusLocally {
        return UpdateStatusLocally(datastore: self.provideResultDatastore)
     }

    public lazy var provideTimeProvider: TimeProvider = {
        TimeProvider()
    }()

    public lazy var provideCMSamplesToVideoRecorder: CMSamplesToVideoRecorder = {
        CMSamplesToVideoRecorder()
    }()
}

As you can see, it's like a large graph that will provide instances when required, and they can be singletons or as many as requested, based on how the provide method/property is written.

So, to obtain a new BroadcastExtensionModel instance we just need to do: CompositionRoot.shared.provideBroadcastExtensionModel(broadcast: Broadcast()) , that, internally will trigger any object instantiation required to fulfill that request.

Chapter II: Then, automate as much as possible.

Analyzing how our composition root looked, we thought that it could be generated automatically, in fact, it was quite easy to add new classes in there, was a matter of copy, paste and replace, so would be great to get rid off all that boilerplate.

Here we use Sourcery a lot, in every project, so it was just a matter of minutes to think: "Hey, we could generate it using Sourcery, dropping some comments here and there and et voilá." For those who don't know what Sourcery does, it is an excellent tool that it's able to generate code automatically using Apple's SourceKit.

It was not that easy.

Not at all.

The closest we got to that was this:

Classes in our project

class Broadcast {}

// sourcery: singleton
class ResultDatastore {}


// sourcery: instance
class UpdateStatusLocally {
    private let datastore: ResultDatastore

    // sourcery: inject
    init(datastore: ResultDatastore) {
        self.datastore = datastore
    }
}

// sourcery: singleton
class TimeProvider {}


// sourcery: singleton
class CMSamplesToVideoRecorder {}

File automatically generated by Sourcery

// Generated using Sourcery 0.10.1 — https://github.com/krzysztofzablocki/Sourcery
// DO NOT EDIT

// With :heart: from Karumi.

// swiftlint:disable line_length
// swiftlint:disable variable_name
class CompositionRoot {

    static var shared = CompositionRoot()

    lazy var provideCMSamplesToVideoRecorder: CMSamplesToVideoRecorder = {
        return CMSamplesToVideoRecorder()
    }()

    lazy var provideResultDatastore: ResultDatastore = {
        return ResultDatastore()
    }()

    lazy var provideTimeProvider: TimeProvider = {
        return TimeProvider()
    }()

    var provideUpdateStatusLocally: UpdateStatusLocally {
        return UpdateStatusLocally(datastore: self.provideResultDatastore)
    }

}

Handmade extension

extension CompositionRoot {
    public func provideBroadcastExtensionModel(for broadcast: Broadcast) -> BroadcastExtensionModel {
        return BroadcastExtensionModel(broadcast: broadcast,
                                       resultDatastore: provideResultDatastore,
                                       timeProvider: provideTimeProvider,
                                       updateStatusLocally: provideUpdateStatusLocally,
                                       videoRecorder: provideCMSamplesToVideoRecorder)
    }
}

It matched all the requirements we have: was super fast, the code was being compiled, we only add comments to our code. It seemed to be really promissing.

Let's drill down all the issues we found that made us reject this approach.

Chapter III: Finally, evaluate and decide what to do when you have all the information.

After a few weeks dealing with Sourcery, we were able to generate a composition root for some pet projects, where a ViewController required a presenter that would interact with a data source to draw something on a view. It was time to confront it with real projects, with third-party libraries, doing more than saying hello, and what's better than our SuperHeros Kata to test this?

Some classes require runtime parameters before being instantiated.

As you can see in the code above, we need a handmade extension for our composition root if we wanna get a BroadcastExtensionModel instance, it requires a Broadcast object, that will be built by somebody else, the iOS runtime maybe. How can we include that in our "graph"? There was no elegant way of doing it.

ViewControllers and Storyboards.

This tools was working fine as soon as you init all your ViewControllers without Storyboards, if you wanted to use them we had to add something like:

// sourcery: instance
// sourcery: as = SuperHeroDetailUI
// sourcery: build = "BothamStoryboard(name: "SuperHeroes").instantiateViewController("SuperHeroDetailViewController")"
class SuperHeroDetailViewController: KataSuperHeroesViewController, SuperHeroDetailUI {
    ...
}

to get automatically this:

var provideSuperHeroDetailUIForSuperHeroDetailViewController: SuperHeroDetailUI {
        let superherodetailviewcontroller: SuperHeroDetailViewController = BothamStoryboard(name: "SuperHeroes").instantiateViewController("SuperHeroDetailViewController")
        return superherodetailviewcontroller
    }

First alert signal

We are adding code in a comment that will be copied and compiled in some other class. That's not cool. That's safe because we have a compiler behind, but refactors are hard, and if you have to write code, write it as a code, not as a comment.

Hierarchy not available.

We relay on Sourcery to parse our source code looking for comments that will generate the composition root right? There is a problem with that, we cannot access the hierarchy of a class being parsed, so if you have a parent class A that requires injection, any child class B will have no way to get more dependencies than the ones found in its declaration.

Showstopper found.

With that problem, there were no chances this could succeed, at least with this approach and set of tools. So, what to do now? Leisurely, gather all the information from the process and study how you can apply it to mitigate the problem you were trying to automate with no luck.

What have we learned here?

First, we love Sourcery a bit more than before; we don't blame it for this, it's an extraordinary tool that just wasn't the right one for what we tried to do. Now we are back to our handmade composition root, we've realized that once you have a bunch of classes in there, the cost of adding new types is quite low.