Agent runs e2e tests instead of a teammate: what do you still keep human?
I used to assign implementation tasks to teammates. Over the last months I've moved most of that to coding agents: once the requirements are confirmed, I press Enter and the agent implements it and runs the e2e tests. If I need proof, I tell it to capture screenshots as evidence and write a short report.
For me, explaining a task to an agent is easier than explaining it to a person and then re-checking the result. I used to think "one person, one project". Now one person can run several, as long as the requirements are written down clearly before anything is handed over.
What I keep doing myself: collecting incomplete requirements (meeting notes, client questions) as requests, and only handing one to the agent when it is complete. When the client changes something later, I add it as a change request on the same item so the agent gets the original context. The screenshot is my own list of such requests (sample data).
What I'm unsure about: this is my own opinion, not something I measured. Where do you draw the line? Which parts do you still give to a human (reviewing e2e results, client-facing decisions, ambiguous specs), and which do you trust an agent with end to end? Disclosure: I'm the maker of PMDesk, so I'm biased toward this workflow.
Replies
Pretty much the same split here. I decide what to build, the spec, and anything that goes out to customers. The agent handles the implementation, tests and verification, and it has to check the real screen before it reports back. The line for me is whether it can be undone: anything reversible it just does, anything that isn't, like emailing a real customer, deleting data or touching production, it has to ask first, and hooks enforce that so it can't forget.
Data lineage still has to be checked manually. An agent passed the suite because the DOM matched, leaving a report with a confident number nobody had measured. It took a full day to discover it had simply validated a hardcoded placeholder.
the part i'd keep human isn't the running, it's the grading. an agent that implements, runs the e2e and writes its own report is marking its own homework.. the screenshots it picked are it deciding what counts as proof. what held for me is it never signs off on a change to the thing it's judged against, if the only way to green is loosening a check it stops and a person looks. when it hands you the report, do you read the actual run or just trust the summary?