Over the last few months we talked to a lot of people involved in building products. We conducted more than 40 interviews with product managers, innovation managers, startups, CEOs and more. The main take-away? Awareness and understanding of the different risks that need to be addressed varied substantially. The one that was either ignored completely or simply accepted as a given was the value risk.

As you're reading this, you're likely involved in building products as well. In this post I'd like to share my understanding of why product is such a risky business but also why I think this can be actively managed. To do this, I will:

• Explain the four kinds of risks that need addressing,
• Summarize the misunderstandings I came across in the interviews,
• Describe strategies to tackle those risks, and
• Emphasize the special role of The One Risk To Rule Them All:Value!

Marty's Four Risks

As I'm currently making my way through an extensive list of the best books on product management, Marty Cagan's Inspired naturally crossed my way, too. My key take-away was his division of product risk into four different parts:

We think of four types of questions we're trying to answer during discovery:

1. Will the user or customer choose to use or buy this? (Value)
2. Can the user figure out how to use this? (Usability)
3. Can we build this? (Feasibility)
4. Is this solution viable for our business? (Business viability)

He describes the main task of Product Discovery to systematically assess those risks whenever they are not clear.

In a previous post I wrote for tarent, I concluded that there are two different types of risk that we can tackle with different kinds of artefacts: Product risk and market risk. They more or less reflect value, usability and feasibility risks, but I simply overlooked business viability and the constraints that an organization may possess that limit what kinds of products we can build, their business model or how we market them.

I like Marty's explanation because it neatly separates the technical aspects (feasibility), the execution (usability) and the demand (value) of a specific product. These insights made me feel like my classification of artefacts in that post was outdated already. Before I give it another try, let's get some more opinions.

"There's nothing you can do!"

Since product validation is something that we aim to provide as a service, we used Mom-Test-like questions in our interviews. The most important part to us was getting an understanding of how the interviewees do discovery. For this, we asked to walk us through the whole product development process from ideation to launch.

There were two repeating narratives regarding validation:

• they simply accept the risk that "9 out of 10 products fail in the market", gather some engineers, raise a budget and start building.
• they buy secondary market research data from external sources, assess market size and competition to estimate revenues, ask engineering what it takes to build it, look each other in the eye and start building.

Some interviewees identified the technical feasibility as the number one risk and a single one explained that they constructed tests by setting up landing pages and measuring signups - only to build the product anyways when too many of those tests showed a lack of demand. To all of them, except the one mentioned earlier, primary user research was synonymous to "testing the UI/UX of the product".

This is by no means representative, but I was still very much surprised by the outcome. It seems like a common verdict regarding validation is that, to quote verbatim, "there's nothing you can do" to assess the value risk but to commit, build and ship.

I am not convinced

In the article mentioned above, I categorized Pretotypes and MVPs (in their traditional flavor) as tools to assess market risk and Prototypes and Proof-Of-Concepts to assess product risk. I want to update this categorization to reflect Cagan's four risks.

Value Risk

To find out whether or not a product (or feature) provides value to a user, we have a wide range of tools at our hands. There are many variations and in-betweens, but here are the two most popular:

A Pretotype is an experiment to measure demand as small commitments like personal information, time or cash, e.g. by leaving a phone number or email on a marketing page. Those experiments are designed to be extremely light-weight in terms of effort and costs. While the reliability of the data is not always optimal, they often result in valuable indications.

Interviews. Done well (e.g. by passing the Mom Test), they can be both fairly quick and insightful. By being qualitative, interviews give valuable hints on the direction the product should be moving towards when the initial assumptions turn out false.

Usability Risk

Usability is usually assessed with prototypes. For digital products, this mostly means so-called high-fidelity prototypes that we put in front of the users to use and simply observe.

Feasibility Risk

This is where we'd try pulling off a rough Proof-Of-Concept of the risky parts over the course of a few days.

Business Viability Risk

Ensuring business viability consists of continually assessing the constraints of the organization. This means that important stakeholders such as Legal, Sales, Marketing, Finance and many more should be taken into account.

Why Value Rules

Building something that in the end nobody's interested in sucks. In an organization, this is the purest form of waste. There is, therefore, no real point in investing time to assess the other risks when value is still questionable. Cagan sums this up as:

If the value is there, we can fix everything else. If it's not, how good our usability, reliability, or performance is doesn't matter. [...] One of the biggest possible wastes of time and effort, and the reason for countless failed startups, is when a team designs and builds a product—testing usability, testing reliability, testing performance, and doing everything they think they're supposed to do—yet, when they finally release the product, they find that people won't buy it.

This clashes with the approaches revealed during our interviews. If assessing value is so important, there seems to be a natural order of product discovery: Value first, then feasibility/usability while gradually ensuring business viability.

There are exceptions to this rule, of course, but still I want to encourage you to give the value assessment extra care. Coming up with meaningful experiments and executing them is easier and faster than it seems at first, and first-hand data unique to your idea is invaluable for avoiding an expensive flop. So let me finish with another quote:

The worst part of this scenario is that, in my experience, it's so easily avoided.