How many users does it take to validate a product?
When you're a first-time founder/maker, and you're the only person using your own product, you probably overlook important things. That's why you need to test the product with as many people as possible.
The problem is that time is limited. You have to decide not only how many people will test it, but also who those people should be.
Yesterday, I launched my Chrome extension for the first time publicly, and my goal is to get feedback from at least 20 testers.
People either describe their experience in writing or I watch them use the product during a call.
My testing group consists of:
Developers and UX/UI designers – people with experience who understand common patterns and expected behavior.
People who have the problem the tool solves – my potential users.
If both groups point out the same issue or suggest the same improvement, it almost always becomes a top priority in the roadmap.
What makes feedback relevant to you?
The number of testers?
The mix of respondents?
Or the way the feedback is collected?
Finding: Developers were able to give me detailed and more accurate feedback than users itself. Developers spotted many things, recommended things while user said: It's okay. :D
Replies
I’m still learning this the slow way, but I’d separate two goals: finding broken UX and proving real demand.
For broken UX, 5–10 people from different backgrounds can reveal a lot. For demand, I’d rather see 3–5 people from the exact painful segment come back on their own or ask to keep using it. A random tester saying “nice product” feels less useful than one target user changing their current workflow even a little.
So 20 testers is a good start, but I’d track who repeats without being pushed.
minimalist phone: reduce your screentime
@grace_lee26 Well, but you need to motivate a user to give you a deeper breakdown... usually a sort of reward, like free access for the whole year or lifetime in the initial stages.
Really thoughtful approach, @busmark_w_nika - especially combining written feedback with live observation. In enterprise B2B, I’ve found that the mix of testers matters more than the raw number: users, admins, buyers, and security or compliance teams often identify completely different risks. Developers can uncover technical and usability gaps, while the people doing the actual work show whether the product solves a problem worth paying for. Clear consent around recordings and sensitive company data matters too.
minimalist phone: reduce your screentime
@mayukh_bit If you know where I can find testers for my product specifically, feel free to ping me on LinkedIn :D
minimalist phone: reduce your screentime
@novamcbo what is your app?
I'd flip it – before counting users, count unanswered questions. My clearest signal wasn't signups, it was that the top Quora question in my category had zero answers and the beauty subs kept a permanent queue of the same request going unanswered for days. Demand with nobody serving it told me more than my first 50 users, because the first 50 are friendly and will use anything you hand them. (disclosure: I'm building siOsi )
minimalist phone: reduce your screentime
@aktivist How many active users do you have at the moment?
minimalist phone: reduce your screentime
@novamcbo Me too. But found some among my friends :)
Hi Nika,
Congrats for launching, I am preparing my launch also soon and I know how busy and chaotic this time can be.
Regarding your question:
Quality over quantity, always for me.
UXers and developers are trained to spot friction and name it, which is why their feedback reads sharper. However, that's also the limit: they're pattern-matching against general UX conventions, not against the actual problem you're solving. Feedback that's sharp but doesn't apply to your target audience isn't more useful, just louder.
Users are the opposite. They often can't name the fix (and they shouldn't), but they know when something's off, even when "it's okay" is the whole sentence. That's not thin feedback. It just needs a different way to draw it out.
Iterative testing is a usual answer for that: a few short rounds instead of one long study. Findings get specific by the second or third round. Template here if you want to try it: https://medium.com/swlh/preparing-your-iterative-usability-study-for-success-incl-templates-4fe866aeae40?sk=1ef218a6a02ee6413f0e1c2d661cd7b2
If budget allows, add a UX/UI audit on top. An audit checks the interface against known patterns. It won't tell you whether the tool solves the problem for the person it's actually for.
minimalist phone: reduce your screentime
@myrto_p Thank you so much. ATM, the product is in the early stages, but I am trying to ask more experienced people beforehand and fix things along the way :)
The number that mattered for me wasn't how many testers - it was realising I'd been counting the wrong thing entirely.
I spent three weeks thinking I had zero validation. Then I checked the analytics instead of my gut: my launch post had 5 views. Not 5 likes, 5 views. I'd been reading a distribution problem as a demand problem, and no number of extra testers would ever have told me that.
So the thing I'd add before your two groups: check that the people you think you reached actually saw it. Views, not responses. It's the cheapest test there is, and it decides whether the silence means anything at all.
On the 20 - I'd care less about the count and more about whether anyone comes back without you reminding them. One person opening it a second time on their own is worth more than ten polite "nice tool" replies, because the polite reply is free and the return visit isn't.
minimalist phone: reduce your screentime
@bharatlearner atm, I have 22 stable users who use it actively, at least the stats are saying that. So it is not so bad for my very first extension :)
@busmark_w_nika 22 people who kept it installed is a much harder test than 22 who tried it, so that's a real number - I'd take it over my zero any day.
The only thing I'd watch is that "stable" can mean two very different things: load-bearing, or forgotten in the toolbar. Both look identical in an install count. Cheap way to tell them apart - did the active number move when you shipped something? If people notice a change, they were using it. If the line stays flat through a release, some of that 22 is furniture.
Not a knock on the number. Just the trap I'd walk straight into if I had one.
There is no one number, because validate is three questions stacked and each needs a different sample.
Does it work is a usability question and five people gets you most of it, because the failures repeat. Do people want it cannot be answered by a count of testers at all, only by the same person coming back unprompted. Will they pay needs money to actually move, and there the interesting fact is whether the number is zero, not how large it is.
What I would worry about more than how many is who. You recruited through LinkedIn, PH threads and DMs. Everyone who says yes to that is highly online, comfortable installing an unknown extension from a stranger, and already inclined to be generous to someone whose posts they follow. That is not a neutral group, and scaling it up does not make it one. Twenty of the wrong sample gives you the same answer as two hundred of the wrong sample, just more confidently.
So the cheap correction is not more testers. It is counting how many of your testers you did not personally recruit. If that number is zero, what you have learned is about your audience rather than your market. Both are worth knowing and they are not the same thing.
On your developer finding, I suspect you measured articulacy rather than accuracy. Developers are fluent in the format of product criticism because they produce it daily. Users are not, so their real reaction leaves through behaviour instead of words. Which is why the two groups agreeing is such a strong signal for you. A user only says it out loud when it is bad enough to overcome being polite.
Of your twenty, how many found you rather than you finding them?
minimalist phone: reduce your screentime
@oshylabs I know that having a big audience is a benefit and users can be biased because
I created it, but I will see whether they will use it long-term. So far, 78 active users in one month or so. It is not that bad.
Regarding "being found" – I am trying to help people with restricted account on X, so maybe that is the place where I am discovered :)
@busmark_w_nika The X angle is the interesting part of that answer, more than the 78. People with a restricted account did not find you through admiring your build in public, they have the actual problem and went looking for a fix. That is a completely different pool from LinkedIn or PH threads, because nobody there owes you politeness.
If even a handful of the 78 came in through that route rather than a direct message from you, split your feedback by source instead of averaging it. The restricted account group's opinion is worth more here, since they arrived with the problem already in hand.
How many of the 78 actually came in that way rather than through outreach?
minimalist phone: reduce your screentime
@oshylabs maybe a very few people came like that up to 10.
@busmark_w_nika Ten out of seventy eight found you without any push on your side, and none of them cost you anything. That is the number worth building around, not the workaround that produced it.
If those ten arrived already convinced they had the problem, weight their complaints above the average of all 78. The DM group tells you who says yes to a stranger. The restricted account group tells you who was already looking.
Are you tracking their feedback separately from the rest, or is it all going into one pile right now?
The recruitment method may be shaping your result more than the sample size is.
You found testers through LinkedIn, PH threads and DMs. That selects for people willing to do you a favour, which is not the same population as people with the problem. A developer recruited that way still gives you something usable, because spotting a broken pattern does not require wanting the product. A target user recruited that way gives you much less, because the main thing you want from them is whether they would use it, and the reason they are in the room has nothing to do with that.
That would explain the gap you noticed without developers being better testers. Both groups are being honest. Only one of them was asked a question their presence has already biased.
The practical split is to stop treating this as one number. Whether the thing works is a usability question and five people will mostly answer it, because the failures repeat. Whether anyone wants it is a demand question, and favour testers cannot answer it at twenty or at two hundred.
Of the people testing now, how many arrived on their own rather than being asked?
minimalist phone: reduce your screentime
@oshylabs I do not know to be honest. At the moment, I am getting traction from my digital footprint. At least in 95%.
@busmark_w_nika 95% from your own reach answers it. That is not a knock on you, most first tester groups look exactly like that. It just means the group already agreed with you once, by following you, before they ever opened the extension.
The fix is not more testers from the same source, it is one cold batch. Find a relevant subreddit or extension directory where nobody knows you and see what five complete strangers do with it unprompted. If they behave the same as your 78, the bias theory was wrong. If they hit something your current testers never mentioned, that is the real gap.
Have you tried a source yet where nobody already knows you?
minimalist phone: reduce your screentime
@oshylabs which source do you mean? I think that I am almost everywhere (besides Snapchat) :D
The number is the wrong axis, and I say that as someone who got it wrong expensively. You can talk to a hundred people and validate nothing, because everybody is polite to a founder on a call. One person who came back on their own, twice, without you prompting them, outranks all hundred of those conversations. So the question is not how many. It is how many returned unprompted. Those are wildly different numbers and only one of them survives contact with reality. The reason people reach for a user count anyway is that it is the only number that exists before anyone has had time to come back. You end up measuring what is available instead of what matters, and it feels like progress while you do it. A signed customer who has not used the thing is not a customer. I learned that one the slow way.
minimalist phone: reduce your screentime
@rabnoor_s the thing is that I do not have ID trackers, so I can't tell at this point who was not interested and what she/he didn't like. 😬