When your AI product says a job is Completed, what has it actually promised?
Nico’s thread this week (green tests, 5.5 hours down) reminded me of a smaller version of the same problem on the B2B AI product I manage.
The scene: a data preparation job finishes and the row turns green with “Completed”. The customer reads that as “checked and safe to use”. It means “ran without errors”. Weeks later someone asks why a model was trained on a version nobody signed off on, and the status page says everything was fine.
Why it’s hard: one status is doing three jobs. It ran. Someone or some check confirmed the output is good enough for this task. Something downstream is using it. Different people decide those at different times, so they diverge: complete and never used, or in use before anyone checked.
What we’re leaning toward on our side (one team’s guess, not a rule): three separate statuses with their own owner, and “checked” cannot be set by the job that produced the output.
-------------------
Two questions:
In your product, what does Completed literally mean, and have users ever read more into it?
If you track “checked” or “approved” separately, who is allowed to set it: the system, the user, or a reviewer?
Replies
we hit the same thing with tool approvals.. a baseline recorded on first sight and one a person approved looked identical, so the warning said you approved it when nobody had. now it says who set it, and approve refuses when it can see an agent running it. do you store who set checked, or only that it is set?