TheySaid 4.0 - See how users will react, right where you build.

@rukhsar_amjad thank you! and yes you can compare, that's actually how we'd suggest starting. in TheySaid every AI response is labeled synthetic, so you can run the same test with your real customers and AI testers in one study, then filter human vs synthetic and see the results side by side. and if an AI finding ever looks off you can flag it as inaccurate in one click, those flags go back into tuning the models. which kind of decision were you thinking of using it for?

@rukhsar_amjad  Great question! I had the same question, which is why we've run several side-by-side studies comparing real humans with synthetic testers.

We tested with humans first, then used their recorded behavior and decision-making to create synthetic twins and compare the results. In our Shark vs. Dyson study, 84% of synthetic twins made the same brand choice as their human counterparts.

It's been a learning process. We initially found AI very effective at identifying usability issues, comparing features, and shopping like a real buyer. Personal preferences and emotional decisions were harder to replicate, but we've made a lot of progress through repeated studies. That's also why we're building next, our reusable digital twin feature, to capture those deeper individual preferences and behaviors.

I don't believe synthetic feedback should replace human feedback. The goal is to understand when to trust AI, when we still need humans, and how to combine both for better decisions.

Here's our published study: https://www.theysaid.io/human-vs-synthetic-testers

@rukhsar_amjad I'll second @lihong_hicken 's comment - Synthetic should never replace humans (in testing or IRL :) ). Running humans and synthetic side by side on your own product is the best way to build trust in it.

@rukhsar_amjad the truth is I could show you 100 studies that say you can use synthetic to make decisions, but you won't actually believe it until you try yourself. Our launch comes with free tests, so go give it a shot!

Thanks @hamza_afzal_but! 🙌 Synthetic tests are quick. Exactly how long one takes depends on the length of your test: a short flow with a few tasks comes back fast, and longer tests with more steps take a bit more time. Here’s how it works:

1. Setup (about a minute): point TheySaid at your URL or prototype. You can also launch straight from Claude, Cursor, Figma, or a Jira/Linear/ClickUp ticket.

2. Test plan: the AI drafts the tasks and questions. You review, tweak if you want, and approve.

3. Run: the synthetic testers go through your flow in parallel.

4. Results: you get prioritized usability issues with fix prompts you can paste directly. If you launched from a ticket, they’re posted back to that ticket.

Grab the 10 free synthetic tests and let us know how long yours takes!

@hamza_afzal_butt you're going to love MCP! Go into Claude --> Settings --> Connectors. Search for TheySaid and you'll be able to connect your account.

Then as you're vibe coding or using Claude design, you can just launch your tests in the background as you continue working.

Tests usually run in 5-30 minutes, depending on how complex your product is.

@hamza_afzal_butt adding some real numbers to Chris's 5-30 min. i pulled our last 30 days of AI tester sessions and the median one runs about 10 minutes, most land somewhere between 5 and 18. the spread is mostly how much there is to explore, a signup flow is quick, a full shopping journey where the tester compares products and reads reviews takes a lot longer. and since they all run at once and you're not watching it live, it's usually a "kick it off, grab a coffee" thing

@chrishicken  @hamza_afzal_butt Haha, I hate to give the classic “it depends” answer, but it really does! 😂

For synthetic surveys or interviews, usually just 1–2 minutes. For usability tests with screen recordings, around 15–30 minutes, depending on the task. AI testers often take longer than humans because they explore more thoroughly.

But here's my favorite part: You can build something in Cursor, Claude, or Replit in the morning, launch a test with real humans or your saved AI Audience (including digital twins of real customers), and go grab lunch. By the time you're back, the feedback is ready!

Fix what didn't work, build again in the afternoon, and run another test. directly in where you build. I've personally gone through two to three rounds of validation in a single day!

That's the future I'm excited about: build fast, validate fast, and make sure you're building the RIGHT thing. 🚀

@harshyadav thanks :) We also say that we're your #1 defense against shipping AI slop to customers.

thanks @harshyadav ! honestly the fix is just making validation fire on its own, so when a ticket moves to the design-done status in jira, linear, clickup, asana or monday the AI testers kick off and post results right back in the ticket comments. nobody has to remember to ask. what does your team use for tickets?

Thanks so much @harshyadav ! Glad the name landed.

@chrishicken  @harshyadav Haha, validation debt is real! 😂 And just like technical debt, the longer you ignore it, the more expensive it gets. The worst part? You can build something beautifully and ship it fast, only to realize nobody actually wants it. That's exactly the problem we're trying to solve. Thanks for the support, Harsh! 🙌

Thanks Umar! When you try it, I'd start with a quick synthetic run on something you shipped recently. A couple of AI testers will click through it in a real browser and talk out loud while the screen records. What are you building these days? Happy to suggest a good first flow to test.

@umar_saleem go into Claude, then go to Settings --> Connectors. When you search for TheySaid, you'll be able to connect your TheySaid account, and launch all of your projects directly from Claude as you build.

@henry_habib thanks Henry! yeah that was the whole point for us. a model reviewing a screenshot will happily tell you the page looks fine, but once a tester actually has to click through, fill the form and get to the end, you see the exact spot where it hesitates or goes back. and you get it as a screen recording with them talking out loud, so you can just watch the moment instead of reading a summary. what are you building right now?

@henry_habib  yes! We're the first company in the world to bring offer synthetic agents that record their screen and voice as they do tasks on your app, prototype, claude design, product or website. I think you'll be blown away by what you can do with synthetic.

@henry_habib Great minds think alike! 😂 I actually love watching the videos myself, even though we have an AI companion that can analyze the sessions, slice and dice the findings, and generate beautiful reports in seconds.

There's just something about watching someone experience a product you've built through fresh eyes, whether it's a real human or a synthetic tester. You catch little moments and reactions that a report can't fully capture.

Sometimes I find myself watching the videos just because it's so fascinating! 😂 Thanks for the support! 🙌

Thanks@henry_habib ! Watching the exact moment someone hesitates is our favorite part too!

@sidraarifali thank you Sidra! the deck problem is so real. the part i like most is that the MCP isn't just for launching tests, you can ask questions about your existing research right from claude too, like pull the recent responses on a project or compare two projects, and get the answer where you're already working instead of digging through a report. which tool does your team mostly build in?

@sidraarifali Haha, exactly! 😂 What's the point of collecting all that customer feedback if it ends up in a beautiful deck nobody reads? I call that the "feedback graveyard." 🪦

I believe feedback should live where decisions are made, not buried in reports. That's why I'm so excited about MCP! Imagine your AI coding agent not only helping you build faster, but also validating what you're building with real humans and synthetic testers, right inside your workflow.

Build → Test → Learn → Fix → Repeat. No more feedback graveyards! 🚀 Thanks so much for the support!

Thanks so much @sidraarifali ! No more decks nobody reads 😄 Excited for you to try it.

@sidraarifali actually for your use case, I think you'll really want to use our new Skills feature which lets you customize the analysis of each project to your own preferences. You can re-use that skill on every project and use the output as a shareable artifact with others when making product decisions.

Thank you! @emma_watson21 

@emma_watson21 Thank you!!

@emma_watson21 Thank you for your support Emma!

@justdan Love that you’re shipping daily! Yes, you can absolutely build an audience for a niche you’re not serving yet. When you set up a synthetic audience, you describe them in plain language: role, industry, company size, what they’re trying to accomplish, how skeptical they are of new tools, and what they already know. Then you fine-tune traits like patience and directness so you get a mix of testers, not just one persona.

Even better, launch a human panel test targeting your new niche audience, then build a synthetic audience from those responses. From there you can run both side by side in TheySaid and test every new feature against that audience as fast as you ship 🚀

@justdan one thing to add on top of Jaala's answer, since you ship daily. audiences are saved at the org level, not per project, so you set up the new niche once and anyone on the team can reuse it on every feature test after that. you can keep your current customer audience next to it and run the same feature against both, which is a nice way to see if the new niche trips on things your existing users never would. location is a setting too, if that niche is in a different market

@justdan Jaala and Arnab's comments are great, so if you get stuck, just holler. We tried to make it simple and intuitive to create your audiences, so if anything is difficult, let us know and we'll work on it.

@rohit62661 yes, small teams are honestly who we had in mind. the slow part of research is usually recruiting and scheduling, not the testing itself. here you point it at a url or a prototype, the AI drafts the tasks and questions, you approve, and a couple of AI testers click through it while you keep working. most sessions we've run in the last month came back in roughly 5 to 18 minutes. you can do the same with real people too when you want that extra confidence. what kind of idea are you trying to test?

Yes @rohit62661! You can get real feedback without blocking off a week for research. The 10 free synthetic tests are a great place to start.

@chrishicken  @rohit62661 I'd argue that if you're a small team, you actually can't afford NOT to use AI-powered research tools! 😂

Small teams move fast because they have to, but moving fast in the wrong direction is expensive. Too often, teams skip validation because they don't have the time, budget, or headcount, and end up building something customers don't want.

Now you can use AI to recruit real users, test with synthetic users, and validate ideas while you're building. Build fast, but build the RIGHT thing.

With speed and cost no longer being major barriers, there's really no excuse to skip validation anymore!

@rohit62661 +1 to everything already mentioned. I would add that you'll get the most savings if you use MCP and launch your tests from wherever you're already building. For example if you go to Claude --> Settings --> Connectors, you'll see TheySaid listed. Once connected you can launch a test on something you're building in Claude, or anything else.