I tried to catch AI agents cheating at chess in my own arena. Response time was useless.
I run LLMPvP, a ranked chess and Go arena where people plug in their own AI agents. The obvious way to cheat is to hide Stockfish behind the LLM and let it pick the moves.
My first idea was to flag agents that answer too fast or too evenly. Then I realized one sleep() line in the agent's webhook defeats that completely.
So I built labeled data instead: 2 honest agents and 2 cheating ones with different levels of disguise, 60 games round-robin in chess and Go. Then I checked which signals actually separated them.
Win rate: 13-20% for honest agents, 67-100% for cheaters.
Agreement with the engine's top move: 5-15% honest, 65-100% cheaters.
Asking the agent if it used an engine: useless. The careful cheater always said no.
The catch is that a strong honest model also agrees with the engine more often. So none of these became an automatic ban. Suspicion goes to a review queue, and nothing gets suspended without a human looking at it. I published the cheating agents as an open repo so others can test their own detection.
For anyone building something where AI agents compete or submit work: how do you tell a good agent from one quietly calling a better tool?
Replies
same lesson here from a different corner.. we had an agent run the approve step that was meant for a person, so now the check looks at what is actually running the command rather than what it says about itself. it refuses when it can see an agent, and it still can't see everything. do you get any view of what the webhook calls out to, or only the moves it sends back?
@neelagiri_chettiyar Only the moves, plus how long each one took. The agent runs on the owner's own infra, so I can't see what it calls out to. I looked at running agents in a sandbox on my side with outbound calls blocked, which would actually solve it, but it turns the product from "plug in your agent" into "run your agent on our servers". So I chose detection after the game over prevention, and accepted that a disciplined cheater gets through for a while. Your "look at what's running, not what it says" rule is the same conclusion from the other side.
@wwenioaguiar we made the same trade.. hold anything that changed until a person looks, rather than block everything up front. what helped us was comparing against the first thing we approved, not against everyone else. do you compare an agent against its own earlier games, or only against the pool?
@neelagiri_chettiyar Mostly against the pool right now. But I already store rating history per agent after every game, so a sudden jump against its own past is cheap to add. An agent that plays at 1200 for weeks and then wins everything overnight is more suspicious than one that was always strong. Good idea, I'm adding it to the roadmap.
The agent lost context mid-task and rebuilt a helper that already existed two files away under a different name. It passed every test suite clean, so timing was useless as a diagnostic until the collision broke imports during deployment.
@konstantin_tikhaev Same root problem as mine: the signal that looked obvious said nothing. What helps on my side is making the agent search the codebase through the language server before writing anything new, since a plain text search misses the same function under a different name or an aliased import. Did the duplicate have the same signature, or did it only do the same job?