TL;DR — Prototypes are created because you need to find the answers to design questions. As soon as you have one answer, it sparks new questions and ideas, and so you may need to iterate on your prototype so that it embodies these new questions for your next test loop. That’s why it’s important to set yourself up for easy iteration so you can go through as many loops as possible when building a prototype.

What is a prototype? Some familiar sayings that crop up are…

“A prototype is a question, embodied”
“A prototype is worth a thousand meetings”

Certainly I don’t disagree with that. We build prototypes because we have questions — There are unknows we need to learn about. So we build something people can see, try out and evaluate. You might say that we embed our questions into artefacts. That’s a prototype, in the UX definition of the word. Engineers have another definition, but that’s not what I’m talking about in this article.

A case story

We built a prototype last month, to test out a concept. It wasn’t a “bad” prototype, it worked out pretty well. But in one aspect I consider it a failure, and that’s what this article is about.

In our prototype, the focus leaned towards making it pretty and of high quality. It wasn’t exclusively focused on that, but it did lean in that direction, rather than leaning towards getting the absolute core right! There is a term which I think may be appropriate to introduce in this context: bikeshedding.

Bikeshedding means focusing on details which don’t contribute to moving your design forward in a substantial way. Another term for this is the Law of Triviality. An example of this phenomenon could be a team discussing the colour of the bikeshed, rather than talking about the layout of the house — even though the layout is much more important, and contains many more unknowns that need to be resolved.

Anyway — In our case we did lean towards focusing on visual quality, which presented us with these problems:

• We spent too much time making things look good
• We started testing late in the process
• Our prototype was not flexible or malleable enough for us to change the fundamentals of the interactions: fundamentally it was hard to iterate
• We ended up running too few tests
• Because of this we had too few and too inconsequential iterations

My reflection on our process

We delivered a great looking prototype and gained the ability to test with users, and thus to learn something about our initial questions. We also learned something through building the prototype, and even came up with new questions during the process. This also presented a problem, however.

Since we did not iterate on our prototype so that it also embodied our new questions, our test yielded less knowledge than it could have done. It was a somewhat shallow test, you might say.

Given how many ressources it takes to recruit users and set up tests, I think this is a non-trivial thing — it’s our job to design our prototypes thoughtfully, so we can optimize them and get maximum bang for the buck when we test.

Learnings from this case story

• We did learn things while building our prototype
• Some questions we were able to answer on our own.
• Our test would have been more efficient, had we been able to iterate / optimize our prototype to embody exactly the questions we wanted to ask in a user test — the most relevant, most pertinent questions.
• In general, we should keep our prototypes flexible and easy to iterate on
• In general, we should keep our prototypes as simple as possible, and avoid bikeshedding
• We should test as early and as often as possible
• We should test with ourselves, but more importantly with people outside the team
• We should remember to always focus on the core, which is to say: we should focus on the stuff that actually addresses your questions.