How to build a simulator for any feature — Part 2: Browser emulation

Sep 21st 2022 engineering

In the previous article of this series, we talked about how unit tests can help us test our functions easily, as well as how getting into the mindset of using unit tests can help us write purer, simpler, and more modular functions in the first place. But what happens when we need to test a system that doesn’t fit in a function? What if we can’t depend on a function’s output to tell us if it’s working? What if our function depends on state set by the UI? In these cases, it’s difficult to accurately model the complexity of how the user will navigate our application — if there isn’t a useful abstraction or simplification that we can test on, we might just have to spin up a live instance of the site and pretend to be a user.

Surely we can’t automate that process, right? It seems too manual and human-brain-dependent to do that. As it actually turns out, the solution lies not in automating anything inside your application, but in automating the input to the browser, simulating your mouse and keyboard so that the application that you’re testing can’t tell the simulator apart from a real human. This process is called browser automation. And what better tool to automate a browser than a headless version of the browser itself — the Chromium project maintains an API called Puppeteer that we’re going to use for this.

Puppeteer — a headless browser

The first step is to get our site up and running with a dev instance, since we’ll want to test these things before we go to production. Since we’re just demonstrating here, I’m going to run this test on Algolia’s production homepage.

Next, we’ll set up a Node.JS program. Choose a logical location for this (probably a single folder inside the root of your project so that you can integrate it with your build process). Inside this folder, you’ll want to create an index.js file that calls all of the tests. This way, just running node test from the root directory of the repo will run all of your tests, and you can bury down into the folder to run more specific ones.

Now, there are some situations where it might make sense for you to make that a post-build command at the end of the build process, but these are going to be more sophisticated tests that you’ll probably want to run manually, so if you’re developing and testing locally, it’s probably best to just leave it there and let your devs manually run node test or node test/specific-test.js when they actually want to run the Puppeteer tests. If your application has a more complex build process that involves less testing locally and more dev instances in the cloud, it might be easier to set up a serverless project (something both Vercel and Netlify excels at), set it to read the tests as lambda functions, and then modify the tests to run when you ping that endpoint. The tests don’t actually need to be run from the same place as the application (since the tests are full-on clients), so you could make a little GUI with some buttons to trigger each test in the cloud.

After running npm install puppeteer, you’ll need some boilerplate code. Here’s the structure around a sample test that’ll log into Algolia as me and get the credentials of some application:

const puppeteer = require('puppeteer');

const login = async ({width, height}) => {
	const browser = await puppeteer.launch();
	const page = await browser.newPage();
	await page.goto('<https://algolia.com>');
	await page.setViewport({
		width,
		height,
		deviceScaleFactor: 1
	});

	// test logic here

	await browser.close();
};

module.exports = async () => {
	login({
		width: 375,
		height: 667
	});
	login({
		width: 1150,
		height: 678
	});
};;

A quick rundown:

1. The first line just imports Puppeteer. Puppeteer is run by the Chromium DevTools team, so it’s essentially a native part of that ecosystem. Importing it here links in a headless version of Chromium (that’s a version that just pretends that it has a GUI), and because it was developed in tandem with Chromium, that version is guaranteed to work with Puppeteer.
2. The first function is called login and that runs our test. We don’t have any test logic in here right now, just the boilerplate that launches the headless Chrome browser, goes to the Algolia website, and sets the viewport to whatever we passed into the function. At the end, we close the browser so it doesn’t stay open in memory.
3. Then, we export a function that runs the same test on different screen sizes.

To run this test from inside index.js in the test folder, we only need to run require("./login")();. With this one-line-per-test setup, we can even group the tests into batches and trigger those batches with flags on the command line, e.g. node test --dashboard. The sky is the limit with the customization here, so tweak it to the needs of your application.

In my case, I’m going to throw errors whenever the tests fail and not worry about batching anything — I don’t have enough tests for it to be meaningful to focus my effort there. In your application though, it might be helpful for these test functions to return the result of the test instead of just throwing errors when they fail, as that’ll give you the ability to print out more detailed messages based on the tests run from index.js. Right now, I’m aiming to make everything run silently unless there’s a problem that we need to fix.

Let’s dive a bit deeper into the actual test logic, which is arguably simpler than what you might expect:

// step 0. accept cookies so the site works like normal
await page.waitForSelector('a[href="/policies/cookies/"] + button');
await page.click('a[href="/policies/cookies/"] + button');

// step 1. wait for the log in link to appear and click on it
if (width < 960) {
	await page.waitForSelector('[data-mobile-menu-button]');
	await page.click('[data-mobile-menu-button]');
}
await page.waitForSelector('a[href="/users/sign_in"]');
await page.click('a[href="/users/sign_in"]');

// step 2. wait for the email input to appear on the new page, focus on it, and type the email from .env
await page.waitForSelector('input[type=email]');
await page.focus('input[type=email]')
await page.keyboard.type(process.env.email);

// step 3. focus on the password input, and type the password from .env
await page.focus('input[type=password]')
await page.keyboard.type(process.env.password);

// step 4. click the log in button
await page.click("form#new_user button[type=submit]");

// step 5. if we're on mobile, we'll need to click one more button to indicate we'd like to stay on mobile
if ((await page.url()).includes("mobile")) {
	await page.waitForSelector('.continue-button');
	await page.click('.continue-button');
}

// step 6. wait for the application selector to appear and click it
await page.waitForSelector('#application-select');
await page.click("#application-select");

// step 7. click on the application with the name in .env
await page.$$eval(
	'div > span.options',
	(options, applicationName) => {
		const matchedOptions = options.filter(option => option.innerText.split("\\n")[1] == applicationName);
		if (matchedOptions.length) {
			matchedOptions[0].click();
		} else {
			console.log(`Application ${applicationName} does not exist`);
		}
	},
	process.env.applicationName
);

// step 8. wait for the api keys button to appear on the new page, and click it
await page.waitForSelector('#overview-api-keys-link');
await page.click("#overview-api-keys-link");

// step 9. wait for the page to load and get the application id and public api key for this project
await page.waitForSelector('#api-keys-page-heading');
const [applicationID, publicAPIKey] = await page.$$eval(
	'input[readonly]',
	inputs => inputs.slice(0, 2).map(input => input.value)
);

// step 10. log the results of our test
if (applicationID != process.env.applicationID)
	throw `Application ID is not correct. Test produced "${applicationID}", but it should have been "${process.env.applicationID}"`;
if (publicAPIKey != process.env.publicAPIKey)
	throw `Public API key is not correct. Test produced "${publicAPIKey}", but it should have been "${process.env.publicAPIKey}"`;

// step 11. close the browser
await browser.close();

This isn’t a trivial example — if you read the comments above each step of the process, none of this should be too surprising. It’s very literally just mimicking the user’s behavior in a real browser, taking into account the quirks of how Algolia’s homepage handles mobile differently from desktop. All that goes along with this is an .env file where I’ve defined the email and password to log in with, the name of the intended application (which is what the user would use to find the right application), and the publicAPIKey and applicationID which the test should return. If all goes well, this outputs nothing. If something breaks though, it’ll throw a descriptive error so that we can jump in to fix it before the build finishes.

So what have we learned in this series? Well, there are two main types of testing:

1. Unit testing — (as discussed in part 1 of this series) this is perfect for simple, pure functions that don’t interact with the outside world. It’s good practice to isolate even the pure parts of otherwise impure functions so that you can test them individually with simple assertions baked right into the same file as the code that you’re testing.
2. Browser emulation — for larger workflows, you’ll need your test to actually pretend to be a user, so we boot up the headless version of Chromium called Puppeteer to “puppet” a fake user and walk through the entire process, with the browser not even knowing that it’s interacting with another program.

It should be noted that these methods aren’t mutually exclusive — this particular functionality of displaying the correct API credentials to the developer in the Algolia dashboard is important enough to test with both methods. We’ve written a Puppeteer program to walk through the entire workflow from the user’s perspective, but in addition, it would be helpful to break down the hydration process of that API keys page that we scraped. If we isolated as many pure functions as possible in that page, we could test them with unit tests to ensure that the vast majority of that process never silently breaks. Since it’s pulling data from the database to display in the view, it’s probably not possible to make the whole thing pure, but that’s the benefit of using both types of testing: we don’t have to get 100% coverage with either method. As long as they cover everything together, we can be confident in our application’s durability and long-term sustainability. Happy testing!