Auriko - Trading desk for LLM calls
by•
Auriko treats LLM providers as trading venues and arbitrages the spread.
Built by ex-quant traders, Auriko’s cost-arbitrage engine calibrates to each user’s request patterns and selects optimized inference paths based on token price, cache behavior, latency, reliability, and request quality.
Auriko benchmarks show average 30% cost reduction against industry peers and direct providers. See the source: https://www.auriko.ai/reports/ll...
Replies
The trader instinct behind this makes complete sense to me. Treating those choices like a live market feels like the sort of thing only people who have lived it would ever think to build.
Agnes AI
@fanny_guillou Exactly - trading and routing, this is the way how you maximize your token usage!
Love that you guys came from the quant trading world and applied real arbitrage logic to LLM routing instead of just defaulting to whatever provider has the shiniest SDK. The benchmarking transparency page is a nice touch too.
Agnes AI
@ece9cgh Thanks for the support! Yes! quant trading and model routing share similar magic - arbitrage and optimize!
How does Auriko handle providers with different caching rules? Some make caching easy to reason about, while others expose less detail. Does Auriko normalize all of that for developers?
Auriko
@withshawn Thanks for the comment! Our data engine tracks and tests provider-specific behavior like token thresholds, block granularity, read/ write pricing, expiration windows, and pricing tiers.
Auriko also handles the provider-specific steps needed to activate caching where possible, like cache directives, cache keys, or session affinity. Developers still see a normal OpenAI-compatible API and consistent cache usage/saving fields in the response.
Dial
the cost angle makes sense but I'd worry about behavioral drift - even at the same nominal price point, different providers running "the same model" can have different quantization, latency profiles, or subtle output differences. if you're routing a request to whichever venue is cheapest at that moment, how do you keep output consistency for something like a customer facing agent where behavior needs to stay predictable
Auriko
@omri_ben_shoham1 Very valid concern, and we are very aware of this concern. Cost optimization only works if quality stays predictable.
We gate the Auriko model catalog and make sure each model is represented truthfully, including quantization. Instead of optimizing for wide inference provider coverage, Auriko optimize for inference provider's quality and credibility. Users can specify the exact model they want, and optimization does not mean quietly swapping to a lower-quality variant. We also evaluate models before adding them to the catalog.
Optimization is about choosing the best provider/path for that model. Under the hood, Auriko computes a composite routing score from live signals across expected cost, TTFT, latency, throughput, and reliability. You can use the default strategy, choose different strategies for different workflows, or specify your own routing weights. We also recommend using separate API keys for different workflows, so our calibration engine can learn each traffic pattern more cleanly and improve cost savings. For example, if throughput matters most for a use case, you can increase the throughput weight.
Dial
the per-workflow API key idea for calibration makes sense, hadn't thought about separating traffic patterns that way. do the routing weights get set once per key or can you adjust them on the fly if a workflow's priorities shift mid-project
SODAX SDK
As someone routing agent traffic across providers, the cost-arbitrage-as-trading-desk framing lands — but the failure mode I'd test first is a venue going bad mid-run. When the cheapest provider starts erroring or its latency spikes, does Auriko fail over inside the same request (retry to the next-best path transparently), or does the caller eat the error and only re-route on the next call? And does the 30% cost-reduction number account for retry spend, since a cheap-but-flaky path can net out more expensive once you add the retries?
Auriko
@hazy0 Yes. Auriko is designed for that failure mode, not just cheapest-route selection.
For retryable failures before a usable response is returned, like errors or rate limits, Auriko falls back to the next high score eligible path.
The 30% cost-reduction number is from real-world usage experiments - retry behavior and provider errors are included. A cheap-but-flaky path is not really cheap. The fallback chain is also logged transparently!
SODAX SDK
Appreciate the specifics — "a cheap-but-flaky path is not really cheap" is exactly the right way to price it. On the transparent fallback log: does it break out per-attempt latency and cost so I can attribute how much of a request's spend went to retries vs the winning path, and can I cap fallback hops per request to bound tail latency when every eligible path is degrading at once?
The quant-desk framing lands, and Michael's point that spend hides in cache pricing and routing more than headline token price matches what bit me. Where I'd push: Ridhwik already asked the quality-floor question, and "optimized without compromising quality" can't be the real answer. I run an LLM backend where the output is the product, and two models at the same token price diverge hardest on the one axis a price benchmark never sees — by the time a complaint tells me, the bad output already shipped. So the mechanism I want to understand: can I define what "request quality" means per route — my own golden set or judge — or is it one internal score you calibrate? Whose definition the router optimizes against is the whole risk surface for anything customer-facing.
Solid
This is so good! We are constantly experimenting with different model providers and from testing this out so far, it's worked great, especially compared to other model routers.
Auriko
@tkeith Thanks Trevor for your support!
Creatium
A 30% inference cost reduction that requires zero change to how our teams build is a rare operational win, and treating providers as trading venues is a genuinely clever framing.
Auriko
@kelly_king3 Thanks!
Smart angle. LLM costs are getting complex fast when you're routing between multiple providers and models. How do you handle latency tradeoffs when optimizing for cost? Sometimes the cheapest call isn't fast enough for real-time use cases.
This is amazing. Love the concept. Thinking of giving this a go but without signing I can't find quantisation of the models. Also a question for you: whats your process if a provider you have on there suddenly swaps to a different quantisation? Can they do it without notice and do you have fail safes for that? I got burnt a little on OpenRouter where a provider I was using swapped to a lower quantisation and I didn't know about it until things started failing. Now I just pin it 3 levels deep to different providers as a fail safe for me.