đź‘€ Do you actually read the code your AI writes?
by•
Be honest...
When the agent says "done", do you read every line?
Or do you just run it and pray hehe
I try to review everything, but with a deadline in 10 days it's tempting to skip :)
So I'm curious:
Do you review AI code line by line?
Do you have tests or another agent check it?
Has skipping a review ever burned you?..
100 views
Replies
Not line by line, no. I read the diff closely when it touches money or auth, and skim the rest.
What I lean on instead is one command the agent can't argue with: type check, lint with zero warnings allowed, a dead-code check and the tests. CI runs the same thing. If "done" didn't pass it, it isn't done.
Money and anything that stores data get read by a human. Copy and layout get a look in the browser.
But, where it breaks: that gate only knows what someone already wrote down. A change that is wrong but valid, passes everything. For that I still have nothing better than reading it.
@hamanovich one command the agent can't argue with, love that :)
do you make the agent run it itself before it says done, or only CI?
@moss_ab_mirande_ney Both
The agent runs it itself before it says "done". It's written in the instructions file it reads every session, so it's part of the job. Then a pre-commit hook runs lint and formatting on whatever is staged, and CI runs the full thing again.
The agent's own run catches most of it, so I rarely see a red CI (almost never). But I don't trust "done" without the CI run, because the agent can pass a check by changing the check. Fixing the test instead of the code is the one I watch for in the diff.
@hamanovich putting it in the instructions file so it's part of the job is smart hehe
almost never seeing a red CI is the dream :)
what does the agent do when its own run fails, fix it alone or stop and ask?..
@moss_ab_mirande_ney Fixes it alone, for the boring failures. Type error, lint warning, a test that broke because the code changed: it reads the output, fixes, runs the whole thing again until it's green
It stops and asks in only 2 cases. One is when the "fix" would be changing a test or a check instead of the code, which is the move I mentioned. And the other is anything that leaves the repo: it can generate a database migration, but I'm the one who runs it, and I do the commits myself.
Not every line anymore.. Anything that touches money, auth or deleting data I read line by line, and a second agent reviews the whole diff against a checklist. The rest I leave to tests. With a deadline in 10 days, I'd keep the money paths on the "must read list" and let the rest go.
@farzam_habibi money, auth, deleting data, that's a solid must read list hehe
does the second agent catch real bugs or mostly nitpicks?..
Not every line. What changed it for me was moving rules out of the prompt and into code. I had an agent with 20 rules in context that could quote them back and still broke them. The ones a script enforced held every time. A rule nothing checks is just a suggestion.