Cekura Bench - Speech-to-speech model benchmarks on live phone calls
by•
Cekura Bench publishes voice AI benchmarks you can verify. Our new speech-to-speech benchmark tests 9 realtime voice models, including GPT Realtime 2.1, Gemini Live, Grok and Phonic, as complete phone agents on live calls: 82 scenarios, three runs each. Models are ranked on reliability, data accuracy, stalled calls, response time and cost, and every call transcript is public. Cekura Bench also covers voice agent benchmarks and STT benchmarks, with TTS benchmarks coming soon.
Replies
Cool stuff!
Really exciting to see this live! The fact that these models were tested on real phone calls with the same agent, prompts, tools, and scenarios makes the benchmark especially interesting. Looking forward to seeing how the results evolve as more models are added 🚀
Happy to see it live 🚀
Voice AI is moving fast, and having an independent way to compare different AI providers on real phone calls is incredibly useful. Really cool work, team!
Really comprehensive benchmark, Voice ai companies should definitely check this out 🚀🚀
Always glad to see more benchmarks for voice AI!
I like that they’re publishing the actual call transcripts with the scores. Way easier to see what went wrong instead of just taking the numbers at face value.
Did the cascade's lead hold on the Medicare intake flow specifically, where it's mostly reading back dates and ID numbers? Curious whether the realtime models lost more on data capture or on the tool calls.