We logged the prompt and the file, but not which model made it
We run 10+ image models plus Veo 3.1 and Seedance 2.0 behind one studio. Until recently every generation record stored the prompt and the output file and nothing about which model produced it. When there was one model that was fine.
With a router it means every quality complaint ends in a shrug. Someone says the face drifted between the image and the video, and I can't tell them whether that was the model, the seed or the upscale step, and I can't reproduce it either because the router might pick differently next time. So the report goes in a doc and dies there.
Writing model, version and the reason the router picked it onto every output has done more for quality than any prompt work we did this year. Not because the models got better. Because complaints stopped being about the product in general and started being about one model in one situation, which is a thing you can actually fix.
The open question for me is whether users should see it. I want to show the model on every output. My cofounder thinks it just gives people something to argue about and moves the conversation off the work. If you route across models, do you expose that or keep it internal?
Replies
Rabnoor, the lineage id is right and we don't have it. Every artifact stores a parent, but nothing carries one job id across image, upscale and video, so the chain only lives in someone's head. On the router I'd keep both fields though. What it saw is the reproducible half, what it decided is what makes a support ticket answerable in ten seconds instead of an hour.
The face drifting between the image and the video is the case your fix still will not catch, and it is the one you led with. Per-output logging gives you a row per artifact. That drift lives in the handoff between two rows, so no single record ever contains the failure. You can know both models and still not see it. What closes it is a lineage id threaded through image, upscale and video, so the chain is one queryable object instead of three rows that happen to share a timestamp. Second thing, and this one bit me: "why the router picked it" is a conclusion, not an input. If the pick depended on load, queue depth, or a price that moved since, then the reason reproduces and the output does not. Log what the router saw, not what it decided.
Dial
expose it, but as a small detail not a headline. put the model name somewhere like a metadata tooltip or an export field, not front and center next to the output. that way the people who care - the ones debugging a bad generation or comparing quality across runs - can find it, but it's not sitting there as an invitation to argue about which model is "better" on every single result. your cofounder's worry is real, but it's a UI placement problem, not a reason to hide the information entirely. you already found that the model info is what makes complaints fixable instead of vague, hiding that from users just pushes the same shrug you used to get internally back onto them.
Gal, this settles it for me. Tooltip on hover, model name in the export metadata, nothing on the canvas. The one place I'd break that rule is when the router switched mid project, because that's the moment someone is about to blame themselves for a prompt they didn't change.
The missing column is a reproducibility problem more than a metadata one, and the rule I would take from it is that the producer's identity gets written at record time, because whatever you plan to infer later from context will eventually be inferred wrong. We learned it from the other end. Our workers identified themselves by reading their own hostname, and inside a container the hostname is the container id, so every restart registered a brand new machine, the fleet filled with ghosts that had done work and then vanished, and the status board sat yellow for weeks with no failing machine behind it. Pinning an explicit node id at deploy time closed it in one line. Yours is the cheap version of that lesson while the log is still short, since a row naming the prompt and the artifact but not the producer cannot be re run or compared against the next one.
@serdarships The ghost fleet is the better version of this because at least yours turned the board yellow. Ours wrote a row that looked complete and wasn't, which nobody investigates. We moved the model and version write to enqueue time instead of on completion, since by the time the file lands the router has already forgotten what it saw. The part I can't fix cheaply is the six weeks of generations behind us with no producer on them, I don't think those are recoverable.