Revert the Fix Before You Trust the Test

After the third time an agent gave me a passing test that was testing nothing, I added one rule: before accepting any fix, I revert it and confirm the test fails.

If the test still passes without the fix, the test is wrong. This has caught assertions against mocks that never change, tests that check the function was called instead of what it returned, and one test comparing a variable to itself.

The agent is good at writing code that looks like a test. It's not always good at writing a test that would catch the bug you actually have. Do you verify your tests can fail?