Guillermo Goñi

One challenge we faced while building an AI testing agent was deciding what it should do when the evidence is incomplete.

A binary PASS or FAIL result looks simple, but it can create false confidence.

For example, a page may display an error message that is expected in one workflow but is a real defect in another. If the agent does not have enough context, forcing a binary result can create a false positive.

We decided to use three verdicts:

I have worked in software testing for more than seven years, and one problem keeps appearing across different teams.

Writing the first automated test is rarely the hardest part.

The real cost appears later.

Selectors change. Test data becomes outdated. Environments behave differently. Flaky failures take time to investigate. Eventually, the team stops trusting the test suite.