How to build a simulator for any feature — Part 1: Unit testing

Sep 9th 2022 engineering

A while back, a tweet came up on my timeline that asked the Twitterverse to scare a product manager in three words. Among the wonderful responses were “one more thing…”, “pretty much done”, “that’s legacy code”, and the perhaps underrated “I tested it”. That last one stuck with me because it’s perhaps the scariest of them all.

It leads to so many more questions: how did you test it? Did you cover all of the edge cases? Was the test accurate to how the piece will actually be used? Did the test only check the expected outcome? Did the test cover if the change may have broken something else in the application? Thankfully, us developers have come up with a good system to answer these questions systematically and thoroughly, so let’s explore the two major types of automated testing in this article.

Unit tests

Before we get started building unit tests, it’s good to take a step back and ask why we need them. In general, we want to make sure that our apps are working! Unit tests are just little pieces of code that verify that things are functioning as they should. To understand how unit tests do this, though, it’s important to first realize that modern development involves modularity (splitting functionality into the smallest pieces possible). Once we chunk up our large tasks into these small functions, many of them will happen to be pure functions: deterministic functions that always return the same output given the same input, without affecting or being affected by anything outside of the function.

Those properties make pure functions straightforward to test, since we can just simulate how those functions are going to be used and check the output against what it’s supposed to be. This is what we mean by “unit test”: using the function we want to test a couple of times with fake data and making sure that we get the correct results. For example, take the famous Sigmoid function, written in Python, with a simple unit test beneath it:

import math

def sigmoid(x):
	return 1 / (1 + math.exp(-x))

def test_sigmoid():
	assert sigmoid(0) == 0.5
	assert 0.9999 < sigmoid(10) < 1
	assert 0.2689 < sigmoid(-1) < 0.269
	assert sigmoid(-100000000000000000) == 0

Sure enough, that last line reveals a bug in our code. Who would have thought that the math library in Python couldn’t handle numbers like -100000000000000000? In hindsight, it’s obvious to me — the positive version we’re feeding into math.exp is slightly outside the range of a double, but I didn’t think about that before! This little bug demonstrates well that we can’t know all of the things that there are to know about our tools. Even though the formula here is technically correct, the tool that I’m using can’t handle the edge case of manipulating such a large number. Thankfully, math.exp will handle this fine if we give it a negative number (which is a positive input into sigmoid, as we give math.exp with -x) because the result just rounds to 0, but in the case that x is positive, we need to write the function slightly differently:

def sigmoid(x):
	if x >= 0:
		return 1 / (1 + math.exp(-x))
	else:
		ex = math.exp(x)
		return ex / (1 + ex)

Now, our unit test returns without setting off any alarms! And if our PM happens to think of an edge case we didn’t, it’s trivial to add another assert to the testing function.

What if our function to test isn’t pure, though? Here’s one idea: refactor whatever you can into a new pure function so that the impure function contains as little as possible. For example, take this JavaScript function, which gets a string value from the DOM before running the Caesar cipher on it:

const runCaesarCipherOnInput = () => {
	const input = document.getElementById("input");
	const result = document.getElementById("result");
	const key = 4;
	
	result.innerText = input
		.value
		.toUpperCase()
		.split("")
		.map((letter, i) => {
	    const code = input.charCodeAt(i);
	    if (65 <= code && code <= 90) {
				let n = code - 65 + key;
				if (n < 0) n = 26 - Math.abs(n) % 26;
				return String.fromCharCode((n % 26) + 65);
	    } else return letter;
		})
		.join("");
};

This works! But it’s difficult to unit test because of all that’s going on. It’s not deterministic (that is, the function doesn’t always return the same output for a given input), and it contains a few side effects that reach outside of the scope of the function. Even the internal loop function inside the .map call isn’t pure! How can we fix this?

Let’s try to pull everything making the function impure out for now, leaving us with a few bits of easily-testable functionality, and then we’ll rebuild the pieces into the original function again.

const caesar = (msg, key, rangeStart=65, rangeEnd=91) => msg
	.split("")
	.map(letter => shiftLetter(letter, key, rangeStart, rangeEnd))
	.join("");

const shiftLetter = (letter, key, rangeStart, rangeEnd) => {
	const rangeLength = rangeEnd - rangeStart;
	const code = letter.charCodeAt(0);
	if (rangeStart <= code && code < rangeEnd) {
		let n = code - rangeStart + key;
		if (n < 0) n = rangeLength - Math.abs(n) % rangeLength;
		return String.fromCharCode((n % rangeLength) + rangeStart);
	} else return letter;
};

const runCaesarCipherOnInput = () => {
	const input = document.getElementById("input");
	const result = document.getElementById("result");
	const key = 4;

	result.innerText = caesar(input.value.toUpperCase(), key);
};

Notice in this example, we still have two impure functions: the function fed to .map on line 3 and runCaesarCipherOnInput. However, these functions are simple enough that they don’t need to be tested! They aren’t actually doing any logic (they both primarily exist to feed pure functions arguments), and the constant declarations and DOM reads are fairly straightforward to understand, so we can safely just expect that they’ll do what we want. I could have gone even further if I used a for loop instead of map, but the point here is that creating unit tests often won’t require a big change from you as the developer. This minor rewrite accomplishes the same purpose, it’s easier to read, and it’s now testable — the important logic bits like caesar and shiftLetter are pure functions, so we can just write assertion tests for them like before:

const testCaesar = () => {
	if (caesar("HELLO", 1) != "IFMMP") throw "Assertion error"; // test basic case
	if (caesar("these are all lowercase letters, so nothing should happen", 8) != "these are all lowercase letters, so nothing should happen") throw "Assertion error"; // test lowercase
	if (caesar(caesar("DECRYPTED TEXT", 19), -19) != "DECRYPTED TEXT") throw "Assertion error"; // test negative keys for decryption
};

const testShiftLetter = () => {
	if (shiftLetter("L", 3, 65, 91) != "O") throw "Assertion error"; // test basic case
	if (shiftLetter("s", 14, 65, 122) throw "Assertion error"; // test wrap around, and custom ranges
};

These functions should go wherever their logic counterparts do! To make it easier to run batch tests, it might be helpful to package the unit tests into logical groups, or even to use a dedicated unit testing framework that handles the boilerplate for you.

But what happens when we’re trying to test something that’s not so easily encapsulated in a function? It’s not as straightforward to test UI elements and entire workflows with simple unit tests. What can we do next? Stay tuned for the upcoming part 2 blog post: simulating the entire browser.