Our AI agent wrote 300 lines of code. It passed tests, but we had to scrap it all.
We’ve been heavily testing autonomous AI coding agents in our workflow. Last week, I asked our agent to implement a new authentication flow. It touched validation, authorization, and our database schema.
Within 4 minutes, it generated a 300 line diff. It also wrote new tests, and the CI pipeline came back perfectly green. On paper, it looked like a massive win.
But when my senior dev went to review it, we had to throw the whole thing out.
The problem wasn't the code itself it was the handoff. The agent gave us a massive wall of code with zero context. We couldn't tell why it chose a specific middleware approach, or why it structured the database write in a way that could eventually cause race conditions. We spent an hour trying to reverse-engineer its thought process before giving up and writing it ourselves.
I realized that a green test suite and a small diff don't actually help me review the code. I need to understand the decisions before I look at the result.
For those using AI agents: What part of an AI handoff actually changes your review decision, and what do you still have to inspect yourself?
Does a summary of the tradeoffs help? A list of the risky paths? I'm trying to figure out how to configure our agent's output so my team can actually trust it.
Replies
Green CI isn’t much of a signal when the agent wrote both the code and the tests, since the tests just confirm its own assumptions. What helps me most is a short note up front listing what it assumed and where it’s least sure like “assumed no concurrent writes here”. That’s how you’d catch your race condition early. I’d still read anything touching auth, schema, or concurrency myself, no matter how good the handoff is.
@hira_maqsood Thats a very fair point, Hira. If a Green CI is grading their own homework it certainly gives you a false sense of security. i love the idea of a confidence summary up front explicitly calling out assumptions like concurrent writes or schema changes is exactly where human eyes should be. I am going to start using that form