Yes, You Can Do Validation Without Coding Knowledge
In this post, I will go through an alternative way of quantitative validation: Skipping the landing page by using a survey instead. This proved to be both fast and useful!
Published March 30, 2022
Last time I boasted about launching a successful Fake Door MVP within less than 3 hours coding time. Well, by replacing the landing page with a Typeform survey, we managed to go live within 2 hours from “let’s do this idea” and “the ad is live”. Welcome to part 5 of my 12 Businesses In 12 Months challenge!
This post will be a quick one, but I will, as always, cover
- what the idea was,
- the decisions we made
- and what I learned.
Can You Really Skip The Landing Page?
That’s what we wanted to find out. And my colleague Josua had the perfect idea for this: Date Night In A Box! The basics are really simple: A user decides on a topic and gets a suitably themed surprise box containing everything she needs for having a great evening.
We expected this idea to be a great testing ground for a different methodology, which is: Get people to click on the ad, reveal that it’s a test (after all, I felt quite guilty the last time) and kindly ask them for feedback through a survey.
By using paid ads for traffic we’d be able to get data on acquisition metrics while the survey would be valuable for both qualitative insights (e.g. regarding existing alternatives, best date night topics or the users’ problems) and skin in the game.
40€ For Ads, Typeform For The Survey
By now our setup is almost habitual:
40€ for ads
spent over 2 days (just for the sake of being able to check the setup)
A (pretty generic) ad creative showing a personalized box, the name and the pricing (39.90€)
Targeting both genders in Germany by the relationship status “in a relationship”
This time, the ad linked to a Typeform survey with 4 main sections:
- Problems: Stack ranking and reasons for the ranking
- Existing alternatives: Stack ranking and reasons for the ranking
- Topics: Stack ranking
- Email signup
We used the Leaner Canvas as an inspiration for our questions for some qualitative insights. Those insights were not meant to be the main data points for this experiments, but rather the conversion.
With a landing page it is easier to tell what a conversion is, but we felt that completing the survey and leaving an email seems to be at least equivalent (if not better). That’s why the conversion metric for this experiment was whether or not people signed up for receiving launch updates.
It was only after setting this up that we found out Typeform limits responses to 10 a month on the free plan, which is absurdly low. The next tier is 25€ per month. Since we want to be as thrifty as possible with these experiments, we just went with the free tier. With average results those 10 responses should be at or near the end of our 40€ budget.
Turns Out: You Can!
Before I start interpreting the results let’s look at some metrics:
- 114 clicks
- 67 survey starts
- 10 survey completions (14.9% completion rate)
- 5 email signups
- 0.31€ CPC
- 35.46€ ad spend before we hit the 10 responses limit, which was when we stopped the ad to save money
- 7.09€ CPA
- 5,755 impressions
- The text inputs were left blank most of the time or responses were pretty obvious
First of all, we were pretty close with our estimation that 10 responses should more or less do the job, which means: Typeform can be sufficient for one such experiment per month. We will start checking out alternatives nonetheless.
The stack rankings confirmed most of our assumptions regarding problems and existing alternatives (with lack of ideas and spontaneous date nights coming out on top, respectively) but enabled us to identify romance and food as the primary topics to concentrate on in possible future iterations.
A CPA of 7.09€ for a business that generates 39.90€ revenue per sale and involves material costs is quite heavy. This was kind of the deal-breaker for us with this idea since it’s not the instant success that we are looking for. The qualitative insights provide ways to probably drive that number down. But compared to our other 4 results, this idea currently ranks on number 3.
Now What? Fake Door MVP?
We will not pursue this idea any further for now. But the obvious follow-up would be a landing page and a suiting ad which both focus on a lack of ideas for date nights. This idea is also eligible to go for a sale directly by including something like a stripe checkout, since the processes after the checkout can all be done manually. So the new hypothesis (that acquisition costs and material cost do not provide enough of a margin for a sustainable business) can be put to the test with minimal risk.
We got what we wanted, though: Skipping the landing page can be both fast and useful and will be a welcome addition to our toolbelt.
As always, I will keep you updated!
Subscribe to my product journey.