UI testing with robots

No Comments

Until recently I was hesitant to recommend UI tests, as far as I was concerned they were brittle and took far too long to write and maintain. This changed after working on a project that used the robots pattern for UI testing, which was inspired by this Jake Wharton talk .

Note: This post applies the robots pattern to iOS with Swift, but it can easily be applied to other platforms in other languages.

So what are robots ? In short, they are just another level of abstraction. Instead of peppering assertions and low level interactions all around, they are put inside “robots” which share a purpose, for example a navigation or a settings robot. Put this way it doesn’t seem any different than a regular helper class, but with some syntax sugar tests become much more readable and maintainable.

So instead of this:

func test_givenDefaultVolume_launch_showsDefaultVolume() {
    let volume = 50
    let app = XCUIApplication()
    app.launchArguments.append("--uitesting")
    app.launchEnvironment["uit-volume"] = "\(volume)"
    app.launch()
    XCTAssertEqual(app.staticTexts["Volume Label"].label, "\(volume)%")
    XCTAssertEqual(app.sliders["Volume Slider"].normalizedSliderPosition, CGFloat(volume) / 100.0, accuracy: 0.1)
}

We get this:

func test_givenDefaultVolume_launch_showsDefaultVolume() {
    launch {
        $0.volume = 50
    }
    playerRobot {
        $0.verifyVolume(is: 50)
    }
}

Note: For these tests I’m using a variation of the “given_when_should” naming scheme, which I find a bit more readable.

The benefits are pretty clear, there is less code overall, it’s reusable and we have a clear context.

context {
    action
}
other context {
    action
    assertion
}

Implementing Robots

In case you want to dive deeper, here is a repository with a working project with various tests. It is a simple audio player (which doesn’t actually play any audio), with an equalizer and a queue. I will be using this project as a basis for the following examples.

Here is a sample robot from the project:

class EqualizerRobot {
    let app = XCUIApplication()
    lazy var volumeSlider = app.sliders["Volume Slider"]
    lazy var bassSlider = app.sliders["Bass Slider"]
    lazy var trebleSlider = app.sliders["Treble Slider"]
}
 
// MARK: - Actions
extension EqualizerRobot {
    func setVolume(to value: CGFloat) {
        volumeSlider.jiggle(toNormalizedSliderPosition: value / 100.0)
    }
 
    // repeat for bass and treble ...
}
 
// MARK: - Assertions
extension EqualizerRobot {
    func verifyVolume(is value: Int, file: StaticString = #file, line: UInt = #line) {
        XCTAssertTrue(app.staticTexts["Volume: \(value)%"].exists, file: file, line: line)
        XCTAssertEqual(volumeSlider.normalizedSliderPosition, CGFloat(value) / 100.0, accuracy: 0.1, file: file, line: line)
    }
 
    // repeat for bass and treble ...
}

Note: The jiggle function is there because adjust(toNormalizedSliderPosition:) is a “best effort” adjustment (according to Apple) and not precise enough for most tests. In my tests it missed +/-10%, so jiggle , jiggles the slider around until the correct value is selected. You can see the full source code for this function in the repo .

Note: file: StaticString = #file, line: UInt = #line is added to all assertion functions and passed to various XCTAssert calls, so that test failures are reported in the test and not in the robot.

As you can see, the robot is nothing special, a few convenience methods for actions and assertions. Even so, it is still important since the testing logic for a specific domain is contained in one place. With robots like this it is easy to refactor when the UI changes or to perform actions needed to get the app into the correct state for another test.

For example, in the test project Player and Equalizer screens share volume values. With robots it’s easy to adjust the volume slider on the Player screen and verify the volume value on the Equalizer screen.

func test_changingPlayerVolume_updatesEqualizerVolume() {
    launch {
        $0.volume = 100
    }
    playerRobot {
        $0.setVolume(to: 10)
    }
    tabBarRobot {
        $0.showEqualizer()
    }
    equalizerRobot {
        $0.verifyVolume(is: 10)
    }
}

Syntactic Sugar

Once you have a robot it is possible to use it like this:

let playerRobot = PlayerRobot()
playerRobot.setVolume(to: 10)
playerRobot.verifyVolume(is: 10)

but that is only marginally more readable than what we started with, so let’s add some syntactic sugar.

First off, to simplify things we will create a base class for all test cases. It will contain convenience accessors for all of our robots.

class BaseTestCase: XCTestCase {
    private let playerRobot = PlayerRobot()
    // repeat for other robots
}
 
extension BaseTestCase {
    func playerRobot(_ steps: (PlayerRobot) -> Void) {
        steps(playerRobot)
    }
    // repeat for other robots
}

Since PlayerRobot is stateless we can store it in a constant to be reused, but if your robot needs to have state remember to recreate them before each test. In complex projects, with many robots, it might be a good idea to create a robots cache which is invalidated after each test.

The function playerRobot(_ steps: (PlayerRobot) -> Void) allows us to use the nice syntax we saw earlier. By passing the robot in an inline closure and thanks to Swift’s shorthand argument naming system we can represent a context where $0 is the robot.

playerRobot {
    $0.setVolume(to: 10)
    $0.verifyVolume(is: 10)
}

You could achieve similar results by requiring all robot functions to return the robot, and then chaining calls.

playerRobot()
    .setVolume(to: 10)
    .verifyVolume(is: 10)

I prefer the former approach, since you can add intermediary steps without breaking the chain or leaving the context.

accountRobot {
    $0.verifyUserLoggedOut()
    let credentials = TestData.credentialsForUser(user: testUser)
    $0.inputPassword(credentials.password)
    $0.inputEmail(credentials.email)
    $0.tapLogin()
}

Test Data

Finally, after creating the robots and adding the syntactic sugar, we can take a quick look at passing mocked data to our app.

In this case the required data is pretty simple ( volume, bass, treble, queue ) so we can pass it through the process environment. Since encoding and decoding queue is a bit more involved, I will use volume as an example. Don’t forget you can check out the repo to see the whole source code.

On the test side we first need to store the volume:

app.launchEnvironment[UITestData.EnvKeys.volume] = "\(volume)"

and on the app side we need to retrieve it and use it where needed:

if let volumeString = env[UITestData.EnvKeys.volume], let volume = Int(volumeString) {
    player.volume = volume
}

Again, to make this process a bit nicer, we can add some syntactic sugar.

First, we create an object that will temporarily hold all of the mocked data:

class LaunchArguments {
    var queue: [(title: String, albumArt: String)] = []
    var volume: Int?
    var bass: Int?
    var treble: Int?
}

and then use it in the launch function of our BaseTestCase :

func launch(_ setup: ((LaunchArguments) -> Void)? = nil) {
    let app = XCUIApplication()
    app.launchArguments.append("--uitesting")
 
    let arguments = LaunchArguments()
    setup?(arguments)
 
    app.launchEnvironment[UITestData.EnvKeys.queue] = UITestData.encodeQueue(arguments.queue)
    if let volume = arguments.volume {
        app.launchEnvironment[UITestData.EnvKeys.volume] = "\(volume)"
    }
    if let bass = arguments.bass {
        app.launchEnvironment[UITestData.EnvKeys.bass] = "\(bass)"
    }
    if let treble = arguments.treble {
        app.launchEnvironment[UITestData.EnvKeys.treble] = "\(treble)"
    }
 
    app.launch()
}

You will notice that this is very similar to the approach we took with the robots. The nice thing about it is that it allows us to selectively mock data, without having to create complex object on the test side:

launch {
    $0.queue = [(“Song 1”, “Art 1”)]
    $0.volume = 11
}

or

launch {
    $0.bass = 50

}

Conclusion

Just because test code won’t be shipped to the end user, doesn’t mean we can’t apply the same principles and care as with actual app code. Doing so makes the end product better and our jobs a bit easier. I hope this article encouraged you to write (at least some) UI tests for your app or, if you already have them, to reconsider if they can be made better.

For a more in-depth explanation of the robots pattern I highly recommend watching the talk by Jake Wharton. For more ways to pass data between tests and app, I recommendthis article by Mladen Jakovljevic.

And as always, don’t forget to check out the full project .