Is it nerfed? We regularly give every model the same coding tasks, track whether its work gets worse, and let you know when it does.
Every model gets the same written prompt and one attempt, working in its own coding tool (Claude Code, Codex or Grok Build) at high effort, with no human help. Each run starts on a clean sandbox. Nothing carries over between runs: memories, sessions, caches and leftover processes are wiped, and the clean state is verified by checksum before and after.
