Same bug, same prompt, three AI coding tools: two wrote the identical fix

A small test worth copying when choosing between coding tools: give each one the same real bug and the exact same prompt, each in its own clean copy of the project, and score them against a checklist written before the first run.

The bug: a Next.js app for YouTube creators rejected some normal YouTube links with "Could not extract a video ID". Claude Code and GitHub Copilot both ran Claude Sonnet 5.5. Cursor ran Grok 4.6, so its result isn't a tool-only comparison.

What happened:

- Claude Code and Copilot produced byte-for-byte the same fix. Same model, same prompt: the tool wrapped around it changed very little.

- A hidden check of 8 link shapes, never mentioned in the prompt, gave all three the same 6 of 8. All three missed embed links and live links.

Three takeaways:

  1. On a small fix, the model matters more than the app it runs in.

  2. It works on the link in the bug report" says little. List the input shapes before prompting, then test them all.

  3. A prompt that names only the failing case tends to get a fix for that case only.

One bug and one run each, so it's a data point, not a ranking.

How would you word the prompt so embed and live links get covered without listing every format?

Full run with the diffs and the scorecard:

Disclosure: from my own channel

13 views

Add a comment

Replies

Best

Your result matches why I switch models rather than tools when I'm stuck. My default is Claude Code, and when it loops I hand the same problem to Codex, Antigravity or Grok. Same idea for anything the model has to judge: a second model checks it. On the wording, I'd skip listing formats and ask it to first write the tests for every URL shape YouTube accepts, then fix until they pass.