Auriko - Trading desk for LLM calls
by•
Auriko treats LLM providers as trading venues and arbitrages the spread.
Built by ex-quant traders, Auriko’s cost-arbitrage engine calibrates to each user’s request patterns and selects optimized inference paths based on token price, cache behavior, latency, reliability, and request quality.
Auriko benchmarks show average 30% cost reduction against industry peers and direct providers. See the source: https://www.auriko.ai/reports/ll...
Replies
Auriko
Creatium
Agnes AI
@ridhwikvinod lol looks like you are a pro on OTC trading! Don't worry about the performance - the cost is optimized without compromising the quality of models!
Auriko
@ridhwikvinod Great question, and we agree. Cost savings are only useful if quality stays predictable.
Quality control starts with explicit model constraints. In Auriko, the model catalog uses truthful model identities, and users can specify the exact model they want, including quantization where relevant. Routing then stays within those boundaries. We also evaluate models before adding them to the Auriko catalog.
Optimization is about choosing the best provider/path for a model. Under the hood, Auriko computes a composite routing score using live signals that represent expected cost, TTFT, latency, throughput, and reliability. You can use the default strategy, choose different strategies for different workflows (we recommend using different api keys for different workflows to maximize our calibration engine's and improve your cost savings), or specify your own routing weights. For example, if throughput matters most for a use case, you can increase the throughput weight.
Is Auriko mainly about monitoring and comparing LLM calls after the fact, or does it help decide where a call should go before it is sent? For developer teams, that distinction matters a lot, especially if they’re juggling quality, latency, and spend across different AI workflows.
Auriko
@crystalmei Great question! Definitely pre-request.
When a request hits Auriko, we build the available routing candidate set, apply hard constraints like capabilities, budget, data policy, parameter support, and availability. The routing engine scores every available candidate i across cost, latency, throughput, and success rate, then picks the best one based on your strategy
The request performance data feeds back into routing: we use it to generate provider health and performance signals, then use those signals to calibrate future routing decisions. So the main value is real-time routing, with observability data used to make the router smarter over time.
Dial
quant background makes sense for this, arbitrage is fundamentally about finding mispriced spreads and providers pricing caching differently is exactly that. the tension I'd want to understand: prompt caching usually rewards staying on the same provider for a session so the cache stays warm, but a router optimizing per-request could bounce a session across providers chasing the best price each time and never let any single cache warm up. does the routing engine account for cache-state as its own signal, like "this provider already has a warm cache for this context, don't move away from it even if a competitor is nominally cheaper this instant"
Auriko
@galdayan Yes! The router accounts for cache state. We also calibrate routing against each user’s usage pattern. For example, if you are a heavy coding-agent user with rapid-fire requests, large context, and long conversation sessions, that pattern becomes an input signal to the routing engine. In that case, Auriko may prefer a provider with deeper cache discount instead of chasing a cheaper provider on headline token price.
Dial
@zxy_action1 that makes sense as a per-user classification. does it ever re-evaluate mid-session though? like if a heavy user's session goes quiet for a while and the cache would've gone cold anyway, does Auriko notice and switch back to price-optimizing, or is the "prefer cache-sticky routing" decision fixed once it's made for that user
Auriko
@galdayan This is a sharp and very deep question!
Yes, the evaluation happens on every request (including mid-session). Cost is computed as the expected cost of the full session, not just the current request. If a cache goes cold, that changes the cost expectation for subsequent requests, and our model accounts for that. For example, if the routing engine chooses provider A for some session, and if cache is warm -- our routing engine will likely stick with that provider. But if the cache goes cold, the equation changes. But regardless of cache-state, we always estimate subsequent cache hit likelihood and conversation length, and compute expected cost for the remaining session.
Dial
@zxy_action1 that continuous re-evaluation makes sense. one more - for a brand new user with zero history, what does auriko default to on the first request? does it start price-optimized until it has enough signal to classify usage pattern, or is there some other cold-start heuristic (like guessing from the client/integration used)?
The prompt caching focus sounds valuable. A lot of teams know caching exists but do not optimize around it.
Auriko
@yolo_xiao Exactly. Caching is easy to know about but hard to price correctly, because every provider handles prompt caching differently, and cost is also a function of the user’s request pattern.
If we had a crystal ball and knew the exact content and timing of a user’s future requests, as well as each provider’s exact caching mechanism, we could compute the cost of the same workload across providers and pick the cheapest path. In reality, the problem is much harder because the information is incomplete. The user’s request pattern has to be estimated, and each provider’s prompt caching mechanism has to be probed and modeled correctly. That is where Auriko’s quant-trading-inspired infrastructure comes in, and how we are able to drive around 30% cost reduction.
Agnes AI
@yolo_xiao Caching is exactly one of the most important factor in model routing - nice catch!
Love the “trading desk for LLM calls” framing. Cost optimization across providers is becoming a real pain point as AI apps scale.
How do you balance cost savings with output quality and latency, especially for production workloads?
Auriko
@longway1 Thanks for the question Kevin! Our data engine actively tracks model performance - latency, throughput, reliability, etc. The routing engine computes a composite score for each available route.
Users can choose preset routing strategies depending on what matters most for a workflow, like cost, TTFT, or throughput. Power users can also set custom routing weights for more fine-grained control.
Congrats on the launch! , Doesn't cheapest-per-request routing fight with caching though? If you hop providers to save on one call you lose the warm cache you built at the last one, and the next 20 calls cost more. Curious if the router accounts for that or just prices each call on its own.
Auriko
@irahimiam Very sharp question! In a nutshell, we model expected cost across the full session, not just the first request.
For example, a heavy coding-agent user with rapid-fire requests, large context, and long sessions will usually have a very different cache-hit profile from someone running periodic deep research. Auriko calibrates routing against each user’s usage pattern, and that pattern becomes an input to the expected cost computation. So in some cases, our routing engine may choose a provider with a deeper cache discount instead of choosing a provider with the lowest headline token price.
BetterClaw
treating LLM providers as trading venues is a genuinely smart framing from people who understand arbitrage. token price differences between providers are real and most teams just pick one model and stick with it out of inertia. the cache behavior optimization is the part i'd want to dig into more, prompt caching can drop costs dramatically on repetitive agent workloads but only if you're structuring requests to actually hit the cache. does auriko handle that automatically or does it require some setup on how you're sending requests?
Auriko
@shubham4real Love this question! Auriko handles the provider side automatically, but we still recommend following prompt-caching best practices since we do not see or log user's prompt (we have a Zero-Data-Retention policy!)
When you route through Auriko, we account for each provider’s caching behavior before sending the request: whether the model supports caching, how cache pricing works, and which route is likely to be cheaper for that workload. For supported providers, Auriko can also apply the right cache hints automatically, like Anthropic cache_control, OpenAI prompt_cache_key / retention, or session affinity where that helps reuse. We also normalize cached-token and savings reporting in the response. The part we still recommend users do is keep reusable context stable: system prompts, tool schemas, few-shot examples, long instructions, repeated RAG blocks, etc.
Auriko
Auriko’s core competence is our quant-trading-grade data pipeline, signal generation engine, and inference cost modeling. We track each inference provider’s prompt-caching mechanics, estimate users’ request patterns, and model inference cost with both provider and user signals in mind.
Our data pipeline also generates real-time signals on provider health, latency, and throughput.
The quality bar for production AI apps is high, so cache aware routing needs good observability.
Auriko
@nicole_h94 Thanks for the feedback! We have detailed log and audit trial for all llm request and all management api key. The goal is to provider enterprise grade control and observability.