Can turning on prompt caching make your bill bigger?

by•

was voice, where one model has two input prices. Today goes back to caching, the pricing side of it.

So what does caching actually cost?

Reading from the cache is cheap. Writing to it is the part people skip. On claude-sonnet-5, a cache write costs about 25% more than sending the prompt the normal way.

Then why bother turning it on?

Because the moment the same prompt comes back, it pays for itself. One reuse covers the write, and everything after that is savings.

So where does it go wrong?

When the start of your prompt changes on every call. Put a timestamp, a user’s name or a request ID near the top, and the cache never matches. Every call becomes a fresh write, so you keep paying extra for a discount that never arrives.

You’re probably wondering how you’d notice, then. You probably wouldn’t, for a while. Caching works fine on day one. A few weeks later someone tweaks the prompt and adds the date at the top. The answers look the same and the tests still pass. Only the bill moves, and it moves slowly.

I only caught the write price because I still read pricing pages line by line, the way you do when it’s all fairly new to you. Fresh eyes have their uses. The discount gets the big font, and the extra charge sits one row below it.

What I do think you should check regularly is two numbers for each feature: cache writes and cache reads. If writes go up at the same pace as your requests, nothing is being reused.

15 views

Add a comment

Replies

Best

That makes the cache read/write ratio worth watching alongside the bill. A prompt change can break reuse without breaking the feature. Do you track that ratio per feature?