đź’¸ Do you track how much you spend on AI?
by•
I opened our invoices this morning... and closed them right away hehe
Claude Max for the whole team, the API on top for our agents, a few other tools here and there...
It adds up so fast!
And the funny part is that a lot of it is the same thing paid twice. Two of us asking our agents to read the same repo, the same files, the same context.
That's actually one of the reasons we started building Murmell :)
So I'm curious:
Do you track your AI spending?
Do you have a budget for it?
Or do you just close your eyes when the invoice comes?..
374 views
Replies
yeah i just dont look too closely, between claude code and a couple apis it's probably more than my actual app revenue some months lol
@jaewon_seo more than the app revenue, that hurts hehe... Is it mostly Claude Code or the APIs? And are you building solo or with a team?
For the AI inside the product, yes, per request rather than per invoice. Every flashcard generation costs the user one credit, and if the model returns nothing usable the credit goes back. That forces you to know what a single call costs, because you're the one eating it when it fails.
The invoice view hides that. A monthly total going up could mean more users or a model getting worse at your task, and those need opposite fixes.
@hamanovich love the credit going back when the model fails, that keeps you honest! And you're right, the invoice hides everything... Do you also track it per model? Like when you switch models, do you see the cost per flashcard change a lot?
@moss_ab_mirande_ney Not on a dashboard, honestly. I measure it when I switch, then stop looking until the next switch
Moving to the newer cheap tier roughly halved the per-card cost on the default path, so the switch paid for itself before I'd even checked quality.
The surprise was the opposite direction though. Pairs where neither language is English go to the expensive model, because the cheap one got translations wrong noticeably more often there. So the cost per flashcard depends more on the language pair than on volume.
@hamanovich wait that's such a good insight, the language pair matters more than the volume! Do you route it automatically or did you hardcode which pairs go to the expensive model?
@moss_ab_mirande_ney Hardcoded. One rule: if neither language is English, it goes to the expensive model. No per-pair list, no runtime routing.
It came out of evals, not intuition. I run a sample of hundreds generated cards past a separate model acting as a judge, and the non-English pairs kept failing on meaning far more often than anything with English on one side.
Spanish to Portuguese probably doesn't need the expensive tier, Bengali to Japanese definitely does, and both get the same treatment. I'd rather pay a bit too much on the easy pairs than ship a wrong translation to someone who can't spot it yet.
the same repo read twice by two people seems like the real leak here. a monthly invoice wont show that, id guess per task tracking would. does murmell flag that overlap while its happening?
@hitensethiya exactly, the invoice never shows it... In Murmell the agents work in the same shared context, so the repo doesn't get read twice, and you see live what each agent is doing. How do you track it today, per task or not at all hehe?
We started tracking it per feature instead of per tool, and the surprise wasn't one big bill. It was the same context paid for again on every single turn, because nothing was cached. The fix there was boring and cut more than switching models would have.
@farzam_habibi the same context paid again on every turn, that's exactly the leak we see too... and yes the boring fix is always the one that works hehe. What did you use for the caching, the provider's prompt caching or your own layer?
@moss_ab_mirande_ney The provider's own caching! The boring part was mostly ordering... the long fixed instructions go first and the parts that change go last, so the automatic cache can actually hit.. For our biggest prompt we're moving to explicit context caching, since three design drafts share the same long instructions.
@farzam_habibi fixed stuff first, changing stuff last so the cache hits... such a simple trick and almost nobody does it hehe. Thanks for sharing :) What are you building by the way?
I got tired of unpredictable bills, so for video rendering I went the other way and run it on GPUs I own. The upfront cost hurt, but now the marginal render costs me nothing. That isn't the right call for everyone, but for any workload you run constantly, it's worth doing the math on owning versus renting.
@holina_factory owning the GPUs is such a different way to think about it hehe the bill stops being a monthly surprise and becomes one decision... for rendering that runs all day it makes a lot of sense :) how long did it take before the hardware paid for itself?
I split it by job, which makes the costs much easier to reason about:
- Claude Code is my orchestrator for anything dev: coding, infrastructure, fixing issues.
- I've got a big database (millions of records) that constantly needs cleaning, categorizing and structuring as new data comes in. For that I use a cheaper, more basic model on OpenRouter. It's plenty for categorization and small jobs, and far cheaper at volume.
- Anything customer-facing (support chat, natural language search in the app) runs on OpenAI.
- AWS Bedrock is a fallback for the OpenRouter models.
- Grok Bot handles everything else, from bookkeeping and partnerships to social and SEO content.
To keep on top of it, one of my Grok Bots sends me a daily digest of AI spend across all services in Slack, so nothing creeps up unnoticed. If you want to be more disciplined, pipe that data into something structured like Google Sheets, Airtable or any analytics platform and chart it over time.
Then at the end of each month I check the total sits well within my margins.
@rich186Â splitting by job instead of by tool makes so much sense hehe, a cheap model for the millions of records is smart... the daily spend digest in Slack is a nice touch :) did you ever catch a cost spike early thanks to it?
@moss_ab_mirande_ney I did actually, just a few weeks back.
Turns out one of the Openrouter hosts went down, and it started routing to a really expensive host instead. So now I’ve implemented a per-token cost cap and a specific whitelist of OR hosts.
@rich186Â oof, silent rerouting to an expensive host is exactly the kind of thing you'd never notice without that digest hehe... per-token cap + host whitelist is a solid fix, stealing that idea :)