If models get smarter, why does the same task cost more?
Not a complaint about pricing. A real question I could not answer until I went looking.
The intuition is that a better model needs fewer attempts, so a job should get cheaper. My bill kept going the other way, and the price per token had gone down, not up.
The answer turned out to be in the docs. Reasoning models produce a block of internal work before they answer, and OpenAI's guide says it plainly: those tokens are "not visible via the API" but are "billed as output tokens". Anywhere from a few hundred to tens of thousands of them. Anthropic bills the same way.
So the answer you read is 200 tokens. The output you paid for is 12,000.
The part that changed how I work: a vague input to a weak model gave a wrong answer cheaply. A vague input to a strong model gives a right answer after the model spends real money working out what you meant. Ambiguity moved from an accuracy problem to a line on the invoice.
Curious whether anyone here has actually measured the reasoning token field across a prompt rewrite. I write about this at slimsnap.ai/blog/smarter-model-costs-more, and I build a screenshot tool, so treat my interest in the ambiguity part accordingly.
Replies
I’d be curious compare the same prompt before and after making it more specific. My guess is the difference could be surprisingly large on tasks where the model has to infer a lot.
SlimSnap
@saman_baloch That is the experiment I want to see too. The fields are output_tokens_details.reasoning_tokens on OpenAI and usage.output_tokens_details.thinking_tokens on Anthropic, so it is a one line diff on a prompt you already run. If you do try it, I would like to hear the numbers.
12,00 tokens for a 200 token answer really puts the cots difference into perspective! 😅
SlimSnap
@_tina_kim It caught me out too. The part that stings is that none of it shows up in the response, so nothing looks wrong until the invoice does.
the ratio is the thing nobody has put a number on yet. 200 visible out of 12,000 billed is 60x, so even a 10x drop in price per token leaves the same task costing 6x more than it did.
SlimSnap
@rabnoor_s Before that number travels: the 200 and the 12,000 were my illustration, not something I measured, so 60x is not verified. OpenAI's own range is a few hundred to tens of thousands of reasoning tokens depending on the problem, wide enough that the ratio swings a lot by task. Your point stands though, and it is measurable, which is why nobody publishing it is odd.
@bickov fair, and my mistake. i took the 200 and the 12,000 as measured and ran arithmetic on them, which is the exact thing i would have flagged in someone else's comment. so the honest version is that the ratio is unknown and task dependent, and both vendors bill the tokens without publishing a distribution.
This is a useful way to explain it. Cost per successful task feels like the metric that matters, not cost per visible token. A smarter model can still be worth it if it finishes reliably in one pass, but hidden reasoning makes that impossible to judge from the response alone.
SlimSnap
@eric_savoie Cost per finished task is the right metric and almost nobody carries it in their head. The awkward bit is that the reasoning count is visible per call, so you can build that metric yourself, you just have to sum it across the retries rather than read one response.