Assumption Testing

Assumption testing is an important skill in all aspects of life, but for Product Managers and Startup Founders it's fundamental to avoiding catastrophe. In the book Inspired, Marty Cagan gave us The Two Inconvenient Product Truths, that most ideas will fail, and the ones that succeed will take many iterations to deliver the benefits. In the book Continuous Discovery Habits, Teresa Torres took this and turned assumption testing into a product science.

The question of what it means to 'Assume' something is a complicated question of Epistemology, or the nature of Knowledge. For this note, we're just going to simplify the definition to:

A belief that may or may not be true, that the validity of another belief, idea, or action depends on being true.

Simplest Example: If I bake peanut butter cookies for your birthday, I might assume you're not allergic to nuts.

When we're talking about the world of ideas, which we do as Product Managers and Startup Founders, there A Few Types of Assumptions that we commonly see:

• Feasibility: Can we build it? (Complexity, cost, resources, logistics)
• Viability: Should we build it? (Profitability, Market Demand)
• Usability: Does it work? (Accessibility, Intuitiveness, Friction, Efficacy)
• Desirability: Does anyone want it? (Value Proposition, Willingness to Pay)
• Ethics: Is it right to build it? (Equity, Justice, Privacy)

Identifying assumptions requires Humility, you have to be willing to entertain the idea that you're wrong. Assumptions can show up when applying common product tools and frameworks like the Ideal Customer Profile (ICP), Value Proposition Canvas, and Eigenquestions. Or higher level strategic frameworks like The Flywheel, Hedgehog Concept, or Porter's Five Forces.

My personal favorite way to identify assumptions is by leaning into Behavioral Science and exploiting our Heuristics and Biases. The Pre-Mortem is my all-time favorite exercise for catching assumptions in ideas and avoiding the Planning Fallacy. In a pre-mortem you're imagining a world where the project has already failed, and asked to explain what happened. Conducting a pre-mortem gives the team the psychological safety to speak their mind and sets the expectation that the effort has already been a failure, you're just trying to explain why.

Assumption tests are tools to de-risk our ideas, but two things must to be true for them to be effective:

1. Success criteria must be clear and measurable
2. Tests need to adequately simulate reality

If the team is not clear on success criteria it can be come easy for confirmation bias win, we will make the results fit our narrative. By establishing what success looks like before we run the test we have more objectivity.

Making sure tests adequately simulate reality can be trickier, because this doesn't mean you always reach for the highest fidelity prototype. You need to understand the nature of what the assumption is, boil it down to it's simplest elements, and test that element specifically. Assumption tests should also be as simple as possible, because speed is essential when the assumptions are endless, so if you can adequately simulate reality with a napkin drawing, and that's the fastest way, then that's what you should do.

Tools for assumption testing with progressive fidelity (and effort):

1. Gather Information: Go ask a stranger, napkin drawings, whiteboard, etc.
2. Outreach: One question surveys, LinkedIn posts inviting feedback, etc.
3. Low-Fidelity Prototypes: Wireframes, Mock-ups, etc.
4. High-Fidelity Prototypes: Functional prototypes, wizard behind the curtain, etc.

There are other tools for assumption testing that can be used when it makes sense:

• For Feasibility Assumptions: Research spike, vendor assessment, experiments, etc.
• When Data is Available: Data mining, analytics, etc.

With these tools you can rapidly test assumptions from the most critical leaps of faith on down until your unmitigated risk falls below your risk tolerance.