Open-source, self-hosted OpenRouter. What would make you try it in prod?
Hi 👋
Jakub here, solo founder from Warsaw. I built GoModel and I'm launching it here very soon. Show HN hit #4 in April.
GoModel is an open-source AI gateway in Go. You can think of it like a self-hosted OpenRouter. It's one OpenAI-compatible API for every model provider, with budgets, caching, guardrails, load balancing, and failover.
In March LiteLLM shipped a compromised PyPI release. Then Portkey went to Palo Alto and TensorZero stopped operating. People are looking for more robust alternatives and GoModel is one of them.
If you use LiteLLM today or are considering an AI gateway for internal use - what would make you try GoModel in prod?
Cost, data privacy, compliance, user interface - or is hosted just easier and you'd never bother?
Follow the page for the launch ping. Happy to answer any question.
Replies
answering the actual question rather than the polite version of it.
what would make me try it in prod is a documented answer to what happens on the day one provider changes a response shape without telling anyone. not an outage, a schema drift. that is the failure that actually takes people down and it is the one gateways usually do not talk about, because handling it means having an opinion about what a response should look like, which is more commitment than most proxies want to make.
the second thing is boring and decisive. i need to know the blast radius of the gateway itself being down. if it sits in the request path with no local fallback, i have swapped provider risk for gateway risk and i have to justify that trade to someone.
self hosted plus go already answers the third question, which is whether i can read the thing when it misbehaves at 3am.
GoModel
@rabnoor_s Great questions and feedback!
1. Schema drift depends on what drifted:
- New field we don't know about -> passed through untouched. Same for choices, tool calls and usage.
- Field goes missing -> you get the response with that field empty. No error. Usage for that call may be under-reported.
- Field changes type -> 502 naming the provider and the JSON error. Failover to the next model kicks in if you set one up.
- Errors hidden behind a 200 get unmasked and treated as errors.
We keep recorded provider payloads and replay them through the adapters in CI. When a provider drifts, we re-record and see what broke. And /v1/passthrough forwards raw provider bytes with none of our opinion applied.
2. Blast radius
When GoModel dies it dies loud: connection refused or 502, not silent.
It's one static binary with /health and /health/ready, a 10s graceful drain on SIGTERM, and a stateless request path. Rolling restarts and multiple replicas behind a load balancer work today. What stays per-instance (rate limits, breakers, sticky sessions) is in the production guide: https://gomodel.enterpilot.io/docs/guides/production
If it's down, point base_url at the provider directly. You lose routing, failover and budgets until it's back.
the missing field branch is the one i would put a warning on. empty with no error shows up as a quiet dip in usage rather than an incident, and nobody goes looking for a number that just got smaller.
GoModel
@rabnoor_s You are right - it should be handled.
I've added a GitHub issue for this: https://github.com/ENTERPILOT/GoModel/issues/931
PS: If you have more suggestions - feel free to let me know here or just submit as a GitHub issue.
@santiagodepolonia filing it as an issue is more than i expected, thanks. one more then, same family. when a provider returns 200 with an empty choices array, is that a success with no content or a failure worth failing over on? most gateways call it success and the caller finds out later.
GoModel
@rabnoor_s Thank you for the feedback again!
Both cases have been handled and cause a failure in the latest release - 0.1.91 .
Feel free to share any suggestion / feature request on our GitHub -> Issues page.