How many users does it take to validate a product?

The mix matters more than the number, tbh. We've gone through this with enterprise pilots where getting 5 really specific users who live inside the problem daily gave us more signal than surveying 50 people with a vague interest. Your dev vs. user split finding makes total sense too, developers catch structural issues while actual users just tell you if it felt okay, which doesn't always mean it IS okay.

Your finding matches mine from the other direction: for our utility apps the magic number turned out embarrassingly small – 5. Not 5 users total, but 5 strangers hitting the SAME confusion point. One confused person is an opinion, five is a pattern. So before every launch we've always "soft-tested" with a slice of our email list, and 30-50 people who actually open the thing beat 500 signups who never come back 😄

The mix matters more than the number, in my experience so far. Twenty people who all shop the same way on Amazon tell you less than five people with genuinely different buying habits.

For Merzify specifically, I care about two different groups that aren't the ones you listed, since the product isn't primarily UI-driven. First: people who've actually been burned by a fake review before; they'll tell you within seconds whether the verdict matches what they already suspected about a product, which is the real test of whether the analysis is any good. Second: people who trust reviews completely and rarely question them, since they're the ones who need Merzify most and will tell you honestly whether the result actually changes their mind or just gets ignored.

Your developer versus user finding matches what I'd expect, though. Developers are trained to find the edge case. Users mostly tell you whether something felt right, which is a different and honestly harder signal to extract, since "it's okay" is often hiding a real problem the person doesn't have language for yet.

The way I'm trying to collect it: not just "does this feel right," but "did the result change what you were about to do?" That second question is the one that actually validates anything for me.

How's testing going now that you're public? Is the 20 target actually giving you the pattern-matching you were hoping for?

thought I validated fix because test went green, until reverted code and it still passed ) test never checked that bug at all. Plus counting users gives false confidence if valdiation criteria cannot fail when wrong

I don’t think I’d use the number of testers as the validation threshold at all. 20 people can find a lot of usability problems, but they can’t necessarily tell you whether you’ve built something people will keep using. I’d probably care more about whether the same behaviour keeps showing up across the right users.

Also interesting that developers gave you more detailed feedback. I’ve noticed technical or product people are often much better at explaining what feels broken, while actual users sometimes just work around the friction and say “it’s okay.” But I’d still be careful about prioritising developer suggestions over user behaviour. Someone spotting a UX problem and someone actually wanting the product are two very different signals.

We've separated the product and target audience tests to alpha and beta. This worked better for us as our product required a lot of data engineering to be ready while the UI/UX could already be tested.

For the private alpha we had 10 people. Mainly UI/UX and product oriented testers. For each tester we did first a product walk through and the participants asked questions during it. Later we had another meeting once they've had a chance to test their product on their own.

The goal of the alpha was to make sure that the product was good enough for initial pilot clients, and for the beta test. It took us about 4 months to fix all the issues and improve the product to be ready for the next stage. During this time we've also had some pilot clients that gave us further feedback.

For beta were going to target around 100 users, all of who we hope will be target users. For beta our goal will be to understand how the users interact with the product, and how well we are able to retain them. Our goal for the beta test will be to get data to understand the usage of the product and even find new features to build (that users are willing to pay for).

Hope this answers your question.

I've found the number matters less than the quality of the signal. Five users who match your exact target and hit a real pain point tell you more than fifty random sign-ups. What's helped me is watching for consistent, unprompted reactions to the same problem rather than polite "this is cool" feedback. Once 3-4 people describe the same frustration in their own words, that's usually a stronger validation signal than any headcount. Curious how others separate genuine pull from friendly encouragement.

@busmark_w_nika I would say 20 to 100 fair enough

Great insights. I completely agree that it's not just about the number of testers, but about having the right mix.

One thing I've learned is that developers are great at explaining how to improve a product, while target users reveal whether the product actually solves a real problem. Watching users interact with the product often uncovers much more than asking, "Do you like it?"

Good luck with the Chrome extension!

Yes! the number of testers is so important to me. I want to get as much feedback as I can since I'm totally bias when it comes to my own product.