Auriko - Trading desk for LLM calls
by•
Auriko treats LLM providers as trading venues and arbitrages the spread.
Built by ex-quant traders, Auriko’s cost-arbitrage engine calibrates to each user’s request patterns and selects optimized inference paths based on token price, cache behavior, latency, reliability, and request quality.
Auriko benchmarks show average 30% cost reduction against industry peers and direct providers. See the source: https://www.auriko.ai/reports/ll...
Replies
V2Fun
I like that the focus is not just more models, but using the right route for each request.
Agnes AI
@tammytan516 Thank you for the support!
Auriko
@thys_beesman Thanks and great question! It's both.
The routing engine decides where the llm request should go before it is sent. For each request, Auriko builds the eligible provider set, applies hard constraints like capabilities, data policy, budget rules, parameter support, and availability, then scores the viable routes accounting for expected cost, latency, throughput, capacity headroom, and cache economics. The cost model accounts for provider-specific prompt caching mechanics and the workload’s reuse patterns, so routing is based on expected request-level economics, not just static model prices.
We then feed the request metadata back into our signal generation engine for future routing decisions. This helps us maintain a dynamic and effective awareness of the routing candidates.
How do you measure quality when routing across models? Do you use developer feedback, user behavior, retries, or some kind of evaluation layer?
Auriko
@phoenixhu We measure metadata such as latency, throughput, error rate, fallback, cache-hit and use that to inform our routing engine. And user can check all information in detailed request logs
Pokecut
Congrats on the PH launch! Modeling request patterns sounds helpful.
Agnes AI
@anthony_cai Thank you for the support! give it a try!
Triforce Todos
Congrats on the launch!
For teams running agents that have really strict latency requirements, can you set a hard ceiling on response time and let Auriko optimize cost within that constraint, or is it more of a balance between the two?
Auriko
@abod_rehman Definitely. There are two ways to control this in Auriko.
1. you can set hard constraints, such as a TTFT ceiling. 2. you can use softer control by increasing the weight of response time (TTFT) in the composite routing score. Both affect the routing decision!
@zxy_action1 Michael... this is jaw-dropping. I am beyond impressed by such a novel yet robust approach to token-spend reduction. My budget loves this!
(my brain, however...? it immediately wants to set about reverse-engineering this mf to tune it towards revenue generation... 😈)
Great work!!
Auriko
@grey_seymour Glad you like it!
Typeless
This looks super useful for teams watching their AI bill climb every month. Congrats!
Auriko
@yuki1028 Thanks!
ReplyMind
Big congrats 🙌 Auriko feels practical and fresh, excited to test how it streamlines collaboration.
Auriko
@moon10 Thanks!
This is a smart wedge most teams are eating unnecessary inference cost simply because provider selection is usually a one time decision baked into the code rather than something dynamic. A 30% reduction is meaningful at scale. Would love to know how request quality is scored in your benchmarks, and whether the savings hold up for latency sensitive production workloads or mainly batch use cases. Excited to see this evolve bookmarking for our team's eval.
A trading-desk framing for LLM calls makes sense. Once teams have more than one model and more than one workload, the real work becomes routing, cost control, and knowing why a call behaved the way it did. The audit trail matters as much as the cheaper token path.
Agnes AI
@krekeltronics Definitely - audit trail just assures you everything is transparent!