Inferock-bench is a local proxy that sits between your app and OpenAI, Anthropic, Gemini, or OpenRouter shaped calls. It captures per-call token usage, failures, and retries, then generates an independent receipt showing what you were billed and how much you're actually overpaying for.
auditing your own inference bill is such an obvious gap 🔍 nice one. seeing much overbilling in the wild yet?
Congrats on the launch @himashwetha_gowda. Good find @fmerian.
Question regarding accuracy, is it possible it might not be able to distinguish a genuinely billable provider failure from valid hidden token usage, such as reasoning, refusal, cache, or tool-call tokens?
@rohanrecommends thank you! Anything that could be legit hidden usage like reasoning, cache, or tool tokens doesn't count as loss, it just gets flagged for you to check. Only what we can prove makes the loss number.
This solves a problem I didn't know I could solve, I always assumed API billing discrepancies were just the cost of doing business. Turns out I was wrong.
Thanks @itohan_blessing_eigbadon , honestly, this is exactly what we told ourselves for months, until we actually looked at the per call receipts and couldn't unsee it. Really glad this clicked for you. Pls run it on your own traffic, come back and tell us what your receipts show, genuinely curious!
I like the positioning here, it's not trying to replace the provider's billing system, just verify it. That's a smarter pitch than "cost optimization," which every tool claims.
@ayla_reynolds Thank you, that's how we think about it as well. Your bill already exists, you should just be able to check it.
I've been burned by silent retries inflating my OpenAI bill before. Having an independent receipt for that would've saved me a painful invoice conversation.
@ramish_saje Ugh, the invoice conversation where you both know something's off. Sorry you've lived that one. Next time at least you'd be the one holding the itemized version.
The independent receipt idea is genuinely useful. LLM costs are still surprisingly hard to audit at the individual request level.
@ashir_murtaza1 thank you! That's exactly the gap we kept running into, you can see the total, but never the audit at the individual request level. Glad it resonates.
The independent receipt idea is really practical. It's nice to have a way to verify token usage instead of relying only on provider dashboards.
@athar_jatoi Thank you! It always felt a little odd that the only record of what we bought came from the company selling it, so now you can keep your own. Glad it's practical!