TrueFoundry AI Gateway
Connect, observe & control LLMs, MCPs, Guardrails & Prompts
2.1K followers
Connect, observe & control LLMs, MCPs, Guardrails & Prompts
2.1K followers
TrueFoundry’s AI Gateway is the production-ready, control plane to experiment with, monitor and govern your agents. Experiment with connecting all agent components together (Models, MCP, Guardrails, Prompts & Agents) in the playground. Maintain complete visibility over responses with traces and health metrics. Govern by setting up rules/limits on request volumes, cost, response content (Guardrails) and more. Being used in production for 1000s of agents by multiple F100 companies!
Interactive
Free Options
Launch Team / Built With
TrueFoundry AI Gateway
@itsarunstark Very correctly said, Arun!
The gateway starts by providing visibility into costs: You can track and control costs across open and closed-source LLMs and MCP Tool calls. It also helps to optimize these costs via semantic caching.
We also have advanced features for cost management on our roadmap, like agent memory management, context compression, and more.
Check out our roadmap here: https://www.truefoundry.com/roadmap
This is one of the strongest interpretations of the MCP standard I’ve seen yet.
While most implementations simply expose tools, you’ve built a managed layer that agents can reliably operate through. With secure tool access, permissioning, and consistent interfaces, this has the potential to become a solid backbone for enterprise-scale agent ecosystems. Really looking forward to seeing where the roadmap goes from here!
TrueFoundry AI Gateway
@nitesh_shakya
You’ve captured exactly what we’re aiming for. Most MCP integrations today stop at exposing tools, but real enterprise agent systems need a governed, reliable, and observable execution layer. That’s why we’re investing heavily in:
Secure tool access + permissioning, so agents only invoke the capabilities they’re allowed to.
Enforcing guardrails, logs, metrics, and auditability by default, which becomes critical once multiple agents start chaining actions.
A shared backbone for agent-to-agent communication (A2A) - something we’re excited to expand on in the roadmap.
As teams move toward production-grade agent ecosystems, having this stable foundation becomes the difference between a cool demo and a system you can actually trust in operations.
We’re thrilled to keep pushing the roadmap forward - especially around agent registry, A2A protocol, semantic caching, and guardrails for MCP servers. Really appreciate the thoughtful note 🙏 Sharing our public roadmap here as well - https://www.truefoundry.com/roadmap
We have been using TrueFoundry for 2 years and a half. I think their managed kubernetes (with the right level of abstraction for a ML team) was already a great product.
This AI gateway has further increased the value of TF for us. We use models from both OpenAI (through Azure) and Gemini (GCP). But TrueFoundry provides us with better observability than either native interfaces. Thanks to the gateway we have:
Detected that the P99 latency of some models was much higher than anticipated, and added backup-models to mitigate this issue.
Tracked why our spend was ~twice higher than expected (and corrected the faulty app).
We're not using much the guardrails and rate limit yet. However, I believe they will prove useful to manage complexity as our team grows.
TrueFoundry AI Gateway
@matthieu_perrinel Thanks a lot for the support for us!
We deployed the AI Gateway fully on-prem in <1 hour.
Secure, isolated, and zero external calls. Good for regulated industries.
TrueFoundry AI Gateway
@hsaxena2 Thank you Himanshu. On prem deployments of gateway is a critical consideration for enterprises.
@agutgutia @nikunj_bajaj @deeptishukla Congrats on hte launch! Curious who the ICP is for the product (as someone without much of a technical know how).
TrueFoundry AI Gateway
Hi TrueFoundry,
I'm a small fish in a large pond (MVP-level startup), but your AI gateway looks like a really convenient option for building and scaling the products like ours. We don't yet have enough revenue to jump to your Pro plan, and your free Developer plan feels more like a preview (e.g. we definitely need more than 10 prompts!). Please, please - consider adding an MVP-scale subscription tier! Some of the early customers you acquire this way will grow big, and we'd love to be one of them.
TrueFoundry AI Gateway
@pavel_fomitchov Thank you so much for the thoughtful feedback - and totally hear you on the gap between the free tier and the Pro plan.
We’re already seeing a lot of early-stage teams using the Gateway for their MVPs, so an intermediate “MVP-scale” tier is something we’re actively exploring. Your note helps us prioritize it.
Its a must to have an AI Gateway while building AI applications & TrueFoundry's AI Gateway has been getting strong traction lately. With security and governance becoming very challenging while building AI applications, looks like TrueFoundry is truly going to win the race here. Thanks for coming up with this amazing launch.