Inferock-bench is a local proxy that sits between your app and OpenAI, Anthropic, Gemini, or OpenRouter shaped calls. It captures per-call token usage, failures, and retries, then generates an independent receipt showing what you were billed and how much you're actually overpaying for.
This is the 2nd launch from Inferock. View more
Inferock
Launched this week
Get free signup credits to use all available models on Inferock.
Bring your own keys: run your OpenAI, Claude or Gemini accounts, see where calls fail and what they cost, and get credits back on eligible failed calls.
Serverless: pay-per-token LLM inference. Get signup credits to start using.
Free Options
Launch Team / Built With
Hi Product Hunt community, I’m Bharath, founder of Inferock.
Every team running LLMs in production pays for calls that came back empty, truncated, timed out, or returned broken JSON. You usually find out from a user, and you can’t prove it to anyone.
Inferock sits between your app and the provider and measures every call against a published baseline for quality, speed and cost. Each call gets a receipt that Inferock produces itself, so it doesn’t depend on the provider’s word.
Two ways to use it:
- Inferock Watch: bring your own OpenAI, Anthropic or Gemini keys. Free. You get the receipts and a monthly loss report. Enroll in the credit program and objectively broken calls accrue credits you can spend on Serve.
- Inferock Serve: serverless, pay-per-token inference through us. This is where your credits get spent.
How credits work: credits are non-cash, expire after 12 months, and are capped at $1,000 in the wallet and $1,000 applied per month. They cover verified, billed, unrecovered objective failures: downtime and timeouts, empty or truncated output, invalid structured output, billing and cache anomalies.
Tell us what the receipt should show that it doesn’t yet.
One thing we’ve learned while building this is that a failed LLM call is not always obvious.
Sometimes the request technically succeeds, but the output is incomplete, too slow, or unusable for the app.
That’s why we’re trying to make the receipt useful beyond just showing pass or fail. Curious what details you’d want to see on it when something goes wrong.
Thanks for hunting @fmerian!
@bharath_koneti This is super cool! Solving a real problem with cost and quality visibility and control, with broad provider support.
@justinmahood Thank you, really appreciate it! Cost and quality visibility you can act on is exactly the goal.
@bharath_koneti
congrats on launch!!
you asked what the receipt is missing. for us, it would be who the call was for.
we run a multi tenant thing so every call belongs to a workspace, and the number our customers want to see is theirs, not our total thing.
can you tag calls with your own ids and group by them, or is it per key? 🤔
This is pretty neat. Can you expand on the Cloud RUn architecture? I see that you host Inferlock.ai inference router but are you using services, jobs, some mix?
@cloudalchemist We use Cloud Run Services for the live router, not Jobs. The router is stateless and I/O-heavy: it validates the request, applies tenant routing and spending policies, proxies to the selected model provider, and streams the result back to the client. We keep one production instance warm and scale out under load.
@rukhsar_amjad Thank you! Invalid structured output is one of the failure types Inferock tracks on each call. The call record doesn't show the exact point. We didn't want to hold any PII's in the failure set!
It's not clear what should happen after that, should we contact Anthropic or openai for a refund for this calls?
Great question @bengeekly For Inferock Watch, we issue non-cash, expiring service credits for verified, billed, unrecovered failures.
Refusals, unbilled calls, and fully recovered costs earn no credit.
The wallet cap is $1,000 and the applied cap is $1,000 per calendar month; the service charge stays cash.
refunds for failed calls is such a good idea lol. how do u verify them?
@petrkovacik our gateway log tracks provider status, failure class, and tokens billed versus delivered for every call, so failed calls are auto-flagged and credited back automatically!
Interesting launch! For the per-call receipt, I’d want to see the expected cost alongside the billed cost, plus whether a retry was billed separately. Does the receipt also show when the provider pricing data was last updated?
@adityaharish2002 Those are fantastic ideas, right now every call states its outcome, 'successful' or a myraid of failure states.
All states record the cost of that particular call. Hope that helps.
The expected cost and last updated pricing would be great additions! I'll pass them on to the team!
If you have any more questions, feel free to ask away!