Startup founders spend lots of time preparing to make a good pitch that would be the magic wand for their product.
But what if I told you that it can be no less important to talk to potential customers without even mentioning the idea? And this situation is way more likely to happen than the dreamed elevator ride with an angel investor. The moment comes when you have an idea and want to find out if it will reach product-market fit. Questions are more important at this point than telling your story.
As a SaaS design agency, we often work with startups on their way to the product-market fit. This initial period is a key moment when little things can make or break the whole business.
But how do you assess product-market fit? The answer is metrics.
Product-market fit metrics
When startup founders ask themselves “How to find product-market fit” (or anything else), the most secure answer is “By measuring relevant metrics”. They include the churn rate and the number of specific surveys.
One of the most popular surveys related to product-market fit is known as the Sean Ellis test. It goes with just one question:
If you want to know some other product-market fit survey questions, read our guide on product-market fit. It contains a few metrics to choose from, but all of them fit to the existing product that already has customers. But what about those that are yet to be launched?
Smart founders start thinking about product-market fit before it even gets to the market. One of the best ways to do this is by talking to users directly. We will talk about this later in the article and share some tips with you on how to prepare product-market fit interview questions. But now, let's discuss why asking the wrong questions might cost you a fortune.
The cost of the wrong question
Asking the wrong question can cost millions. It is not an exaggeration, but a real story. In 2009, Walmart decided to make a redesign in order to de-clutter their shops. To find out whether the user would like it, they ran user research.
The question they asked their customers was “Would you like Walmart aisles to be less cluttered?”. Of course, most people said yes. So, Walmart invested millions in the redesign, convinced that it would make customers happier. The result was a large decrease in sales (maybe customers actually became happier, but that’s not what you care about when the sales plunge). Walmart lost over one billion dollars.
Now that we look back at the situation, it seems obvious that the question was driving biased answers. So, how do you pose the right question when you want to verify an idea? There are some rules that you can rely on to check the quality of the questions.
Mom Test
When you come to your mom and tell her a new exciting idea, asking what she thinks about it, what would be her answer? Great idea, darling, I'd love to see it! Is she lying to you? No. Should you base your business decisions on it? No once again.
It does not mean that your mother can't give you a good business advice. But the secret here is to shape the conversation in a way that would not be about praising your idea.
This is a key idea behind the mom test invented by Rob Fitzpatrick, who even wrote a whole book on it. Thanks to this test, you can turn any conversation into a useful source of information to get to product-market fit.
Imagine you have a genius idea: make an app that would connect dog groomers with clients. You come to your mom and ask her if she likes the idea. She says “of course, what a great idea, darling”.
But the conversation doesn’t end there. Next, you ask if she would use this app to find a groomer for her terrier. And if the answer is yes, you ask her if she would buy a monthly subscription. Of course, your mom will be willing to pay a good price for her kid's app! It doesn't mean though that other people will do so.
Naturally, moms want to support their kids and therefore they give compliments to their ideas. The challenge is to lead the conversation in a way that would give really useful information and not just compliments.
The mom test teaches us rules to make interview questions unbiased. Here are the main ones:
1. Talk about the problem, not the suggested solution
Shift the focus from your product to the customer. It is a bit counterintuitive because we really want to know whether our idea is good or not and whether people are willing to pay for it. And asking questions about the problems your customers is nowhere near as easy.
Still, asking about the solution is OK when it is not put in a hypothetical way (Do you think a new productivity app would solve your procrastination issues?), but refers to real experience (Have you tried some methods to fight procrastination?). It brings us to the next rule.
2. Ask about the past, not the future
Here is a real-life story. When people who just bought a treadmill were asked how often they plan to use it, they said 3 times a week. After one month, it turned out that they used it about once a week. They were asked again and the answer was “five times a week” this time, as customers hoped to to recover lost time.
Humans tend to imagine the future way more optimistically than it really is. These pink glasses help us survive in this not-so-optimistic world. That is why you can't rely on what people predict about their future behavior. They are not lying, they are just being optimistic.
3. Avoid compliments
When you hear something like "your product is a great solution", it's time to move the conversation in another direction. Compliments are a symptom of the “mom bias”. The interviewer has to carefully return to talking about customer experience instead.
Coming back to our example of a dog grooming app, how could you shape your questions? First of all, you would start asking about her experience instead of telling the idea.
This question asks about the past. Then, you can find out where she got the info about her groomer. Did she google it or was it another dog owner’s recommendation?
Further you might find out that Lucky hates car rides and the only reason why your mom goes to a dog groomer is that they live on the next street. That's when it becomes clear your mom wouldn't use the dog groomer app.
That’s it, your mom just ruined your idea… Now you don’t rush into it and do more research before investing in the development. Also, you can try looking at other kinds of customers: for example, those attending a dog competition, where people are very serious about grooming.
4. Listen more, speak less
Getting the interviewees to talk is hard, but talking as little as possible can be even harder. It is commonly believed that 90% interviewee / 10% interviewer speaking is a good distribution. How do you get there? Don’t interrupt, even if they start speaking of something that is not very relevant. Ask open questions, not yes/no.
5. Pay attention to emotions
When interviewees show excitement or annoyance about the things they are talking about, ask more questions about their emotions. At the same time, you should be sensitive to topics they don't want to talk about and not push in that direction. Interviews must not be uncomfortable.
6. Don’t ask about prices
This goes back to the rules mentioned above: people can’t predict their own behavior and they want to compliment you. In an ideal world, they would pay a good price for useful products. In reality, most people are not willing to pay more than the minimum price.
What if top managers of Louis Vuitton were asking clients “How much are you willing to pay for a bag?” They would never get to where they are. We are very far from luxury bags, but you get the idea. Don’t base pricing decisions on users.
7. Organize the process
Don’t think of it as “just a talk about our product”. Now that you see how considerate you should be with every question, you understand that the best way to make it is by properly writing all the questions down and trying to follow the script.
Use whatever instruments that can help you: recording devices, note-taking, and so on. Make an exception only for those occasional situations when you meet a customer at a conference or see your mom at a Christmas dinner, trying to get some valuable information about your product.
For more tips on organizing user interviews, read our article “How to talk to users”.
Good Questions
Now that you know all the rules, preparing questions becomes both easier and harder. Here are some examples of questions that can be included in the mom test. Take this list as an inspiration and make your own.
- What does your typical day at work look like?
- Tell me about the last time you faced this [problem]?
- What have you tried to do to solve [this problem]?
- What is good/bad about the solution you are using now?
- How did you find this solution?
- Have you looked for alternatives to this solution?
- How much did you pay to solve this problem?
To make the most of the interview, finish it with a plan for the future (yes, at this point you can talk about your product and about the future).
- Would you like to be on the list of beta testers when we launch?
- Could we meet next week to hear your opinion on our product?
Bad questions
If you already have a draft interview script, make a fast check that your questions don’t look like these:
- How often do you fail to do [a task]?
NO — this question makes the interviewee feel guilty
- Would this [product] be useful to solve this [problem]?
NO — this question is begging for an affirmation and a compliment
- Would you be willing to pay a bit more than you are paying now to get a much faster solution?
NO — the answer will never predict the future.
How do you know that the interview was unbiased?
You never know for sure. That is why, whenever possible, you should use different types of user research to prove your hypothesis right or wrong. For example, in the case of Walmart, an A/B test would do the job.
Curious to find out what is out there beyond user interviews? Read our article about UX research methods.
.png)
