Most design or innovation projects start off the same way: you have a lot of uncertainty about a lot of things and gradually you reduce this uncertainty — perhaps through research, testing or prototyping.

But how can you and the project team obtain a comprehensive overview of the assumptions that you are working with? And how do you communicate the progress being made to outside stakeholders? Introducing: Mythbusters.

Why bother?

Assumptions and hypotheses, both implicit and explicit, accompany product development at every stage. Getting some of these assumptions wrong could mean the difference between success and failure for your product or venture.

By creating a shared understanding of hypotheses within the entire team, everyone will gain a more comprehensive idea of the direction that development and user research is (or should be) heading. Therefore, there’s no longer a need for internal, subjective debates on allocating resources, as everyone has a shared understanding of resource management; “Why the hell are we spending resources on doing this? Oh, that’s why.”

As a project evolves, so do the hypotheses, which is why it makes sense to have a living document that charts their evolution and can demonstrate the progress made in validating or disproving them.

Enter Mythbusters

The Mythbusters document is essentially a list of hypotheses that surround a particular project. There are many ways to generate hypothesis statements, so choose your favourite. Once you have your hypotheses, you can put them in a spreadsheet and give them each an identifying number:

Not all hypotheses are created equal — some are more critical to the outcome of a project than others — so it makes sense to also prioritise them. We usually prioritise them from 1 to 3, with 1 being highest priority and 3 being lowest, but you can use pretty much any scale you want. When doing this, you can try to ask the question “How bad would it be for the project if we get this hypothesis wrong?” — if the answer is “it would very probably kill the project”, then it’s a 1, if it’s more like “it might mean this major feature does not really make sense any more” then it’s a 2, and if it’s “would be annoying, but it’s not really a huge deal at this stage” then it’s a 3.

OK, now that you have listed and prioritised your hypotheses, it’s time to start rating them based on how confident you are about whether they are true or false. You do this by creating 5 columns labelled:

Definitely False
Probably False
Keep Digging
Probably True
Definitely True

Even before any major research has begun, the product team will probably have some thoughts about where on the spectrum each hypothesis lies. It can be a fun exercise to put an ‘X’ in one of the columns for each one based on the gut feeling of the team just to see how this changes as the discovery and validation phase progresses.

You should now have something that looks like this:

The next step is to set up other columns that might be relevant to the project. I find it’s often useful to have the possibility to add notes, which validation methods you intend to use, which personas it’s connected to and also links to research sources (like interview videos or transcripts in Dovetail or UserBit) — but you can add wherever makes most sense for your project — go nuts!

The final step is to could how many hypotheses you have in each stage by using the formula:

=COUNTA()

at the end of each column.

Once you have done all that, you might have something that looks like this:

Congratulations! You’ve just created the first version of your Mythbusters document.

Managing the Moving Hypotheses

As Discovery progresses, the Mythbusters document should serve as a single source of truth for the team when deciding where to focus research and where there are still significant gaps in knowledge which lead to critical uncertainties.

Part of every research synthesis phase, such as collecting the findings from a round of user interviews, should also include the step of updating your Mythbusters document. If a hypothesis was addressed in the research (and honestly, at least some of them should have been) then their status should be updated. Did the research indicate that the hypothesis was true? Move the X one step to the right — if the indication is that it was false, move it to the left.

Leveraging this methodology, we start to get a more organic overview of how our knowledge and certainty is increasing.

In many cases, your research will lead to more questions and hypotheses — that’s great! Just add them to the list and keep the process going for them.

While many business stakeholders usually delight in spreadsheets, it often makes sense to present the ongoing results in a more visual way. This is something we are experimenting with here at MVP Factory and currently use something like this:

The statuses are sized by quantity and then moved along the axis (the numbers below refer to the IDs in the spreadsheet). Maintaining clear and open communication is important for product teams, as it help avoid the Stakeholder Stress Gap.

The Stakeholder Stress Gap

Product Design is often a chaotic, non-linear process particularly in the early days of Discovery and Prototyping. Teams tend to limit the visibility into this chaos (intentionally or unintentionally) to outside stakeholders until there is greater certainty about outcomes which they can report with greater clarity and confidence.

These outside stakeholders can see that something is being done (after all, the team is working hard, and presumably charging for their time) — and assume that the progress is fairly linear in relation to time.

Thus, the design process feels different for outside stakeholder and leads to the Stakeholder Stress Gap:

In order to reduce the Stress Gap, product teams can use regular check-ins using visual and concrete frameworks such as Mythbusters to help adjust expectations and visibility into actual progress. And at the end of the day — reducing stress for everyone should always be a desirable design outcome.

I will be writing in more detail about the Stakeholder Stress Gap in a future post, so make sure you subscribe to stay in the loop.

Thanks to Mason Adair https://www.linkedin.com/in/mason-adair/ for showing me the basic Mythbusters concept and to Leland Peterson https://www.linkedin.com/in/leland-peterson-7630b19/ for introducing me to the Stakeholder Stress Gap.

Daniel Westerlund is a Strategic Designer at MVP Factory in Berlin. If you’re interested in learning more about building digital products or ventures check out the MVP Factory website or on LinkedIn.