OpenGauntlet grew out of roughly a year of research while I was building Imprynt, a voice-first companion. General benchmarks helped me understand what models could solve, but not which local model I would actually want to spend time talking to. I began giving each model the same multi-turn conversations, scoring both rubric dimensions and pairwise preferences, and publishing the replies so the results could be inspected rather than taken on faith. I am sharing the research publicly through D3velop because it seemed more useful in the open than scattered across project files. The Listening and Voices sections are sourced surveys rather than in-house benchmarks and are labeled separately. I would especially value feedback on the evaluation design and suggestions for the next model to put through the gauntlet.

OpenGauntlet

Find the local AI model you would actually want to talk to

Find the local AI model you would actually want to talk to