During my Kartenhaus journey I had to learn the value of a proper commitment during validation the hard way: We needed 2 pivots to actually understand when something that users say is actually reliable. In this blog post, I want to explain:

1. Why commitments are necessary
2. and what kinds of commitments are suitable to gauge demand.

Skin In The Game

Alberto Savoia uses the term Skin In The Game in his book The Right It to evaluate the reliability of validation experiments. Skin In The Game loosely means having a stake in the matter and supposedly has its origins in gambling (like throwing animal skins in the pot at a game of poker). His approach of pretotyping is already data-driven, but he stresses the importance of the data pointing to demand in a meaningful way. And that means that the data must show that the demand is genuine.

Savoia is not the only one with this kind of view. Rob Fitzpatrick basically dedicates a whole book, The Mom Test, to this kind of problem. He shares methods on how to conduct interviews that make it impossible for interviewees (not even your mom) to lie to you when it comes to how much value your idea would provide for them. He agrees that a commitment is the best indicator for genuine demand.

In short: Without a commitment your assessment of value risk is almost meaningless.

What commitment, tho

Taking Savoia’s and Fitzpatrick’s views into account, there are three kinds of commitments that can be used in early product validation: Personal info, time and cash. Let’s talk about each of them.

Personal info

It may seem obvious, but when visitors visit your page and leave their personal information, this can be a commitment. But it has to be genuine information and, if compatible with the experiment, should enable a qualitative follow up, e.g. a phone call.

But not only the user’s own personal information may count as Skin In The Game. Especially for qualitative validation, an introduction to or contact information of another person can count as such. By giving us other people’s information, the person puts her social reputation on the line for us. That’s a strong commitment!

Time Spent

When people are willing to spend their valuable time on learning about the product, this can also count as being genuinely interested in it. This can be the case for e.g. online demos, phone calls, meetings or watching lengthy videos. Done right, this is a powerful tool for both online and offline experiments.

Cash

A cash commitment is the crown jewel of commitments and works in both quantitative and qualitative environments. If people wave with their checks for the promise of using something in the future, that’s a great sign. The applicability varies a bit, though. It may be legally challenging to just add a Stripe checkout to your pretotype landing page that provides nothing in return. For wizard of oz MVPs, where a service is performed manually behind the scenes, this may be a viable form of validation, though. Cash commitments shine in B2B environments where interviews and negotiations are done in person.

Long story short: Cash is king.

Score your commitments

To compare different setups or iterations, it may make sense to create a scoring for each run. I like Savoia’s approach of assigning points to the commitments. A verified, genuine piece of personal information counts as 1 Skin In The Game, as well as 1 minute of time or 1€/$ given as a cash commitment.

The actual assignment may vary (e.g. I personally weigh phone numbers higher than mails and cash commitments may depend on the currency), but this way the experiments get a lot more comparable.

Commitments aren’t everything, but without them, everything is nothing

I had to learn this the hard way. When founding my own startup, it took 2 pivots to actually understand the necessity of commitments. Before the first one, we cared only about our idea, not the users. Before the second pivot, we developed our software together with the users. This worked fine until payday, when people started turning away (”I helped you build this, now you want me to pay for it?”). This painful experience taught me the importance of having commitments first before investing time and money into products that don’t provide genuine value to users.