AI Observability by OpenObserve - OpenTelemetry-native observability for agents and LLMs

@asherraph This is very useful feedback! You're right that the cleanest path today is OTel. If the harness emits OpenTelemetry, you point it at us and you're done. Claude Code and Codex work this way. Your own app works with the auto-instrumentation.

Harnesses that don't emit OTel and don't expose hooks are the gap you're describing. The workaround right now is sitting between the harness and the model with a gateway or proxy that emits spans for every inference call, so you get coverage even when the harness isn't cooperative. A first-class version of that, where you point the harness at an OpenObserve endpoint and it just works, is exactly the direction we're pushing.

BTW, which of the three matters most to you? That helps us prioritize where to start.

@asherraph You're right, and it's the sharpest version of a gap two other people hit today.

Right now it's OTel from the harness, which works when the harness cooperates — and plenty don't. The proxy idea is the general answer: sit in front of the provider endpoint and capture every inference regardless of what's calling it. We don't ship that today.

The trade-off worth naming: a proxy gives you complete coverage but loses structure — it sees the calls, not which agent, which tool, or which session they belonged to, unless the harness passes something through. OTel gives you structure but only where the harness cooperates. You end up wanting both, because they answer different questions.

Someone asked about Hermes specifically this morning and I opened an issue for it: https://github.com/openobserve/openobserve/issues/14410

Which harness is blocking you most right now? If it's OpenClaw or LobeHub I'll file those too.

@sara_dph Yes — that's the case it's built for.

The agent graph gives you the shape: which agent called which, where the tools and models sit. The session view gives you the sequence — turn by turn, with the input and output captured on each span, so you can see exactly what one agent handed to the next. That handoff is usually where things start going off the rails: the second agent got a truncated or badly worded brief, and everything after it is downstream of that one moment.

Repeats get flagged even when nothing errors, which is the nasty case — no exception, no timeout, just the same step eight times.

What does off the rails look like in your setup — wrong tool, or wrong plan?

@sara_dph Yes , You can trace agent-to-agent interactions, tool calls, and LLM calls in the same trace, so when one agent starts going off the rails, you can drill into the span hierarchy to see where the behavior changed.

It’s OpenTelemetry-native, so you don’t need a separate telemetry pipeline. We have integrations for frameworks like OpenAI Agents, AutoGen, CrewAI, LangChain, and more.

@dmoniz22 No first-party Hermes integration today — being straight with you. But two pieces make it workable right now.

OpenCode is already a one-command setup on our side, so the delegated half is covered. And Hermes can emit OTel via the community hermes-otel plugin, which ships traces, metrics and logs to any OTLP endpoint — point it at OpenObserve and the coordinator layer lands in the same place as everything else. We haven't tested that combination ourselves yet.

The seam worth watching is the one you're actually asking about: for a delegated task to show up inside the parent run rather than as its own separate trace, Hermes has to pass a session id down to OpenCode.

This is a good catch and we'll add proper Hermes support — I've opened an issue for it so you can follow along: https://github.com/openobserve/openobserve/issues/14410

Are you delegating to OpenCode from a single Hermes instance, or one per project?

@dmoniz22 Hengfei has the integration path covered, so I'll just add one thing. The "hidden layer" you describe is exactly what we hear most. Costs drift and behavior degrades quietly, and by the time someone notices, nobody can say which project or which delegated task did it. Getting coordinator and delegate usage into one place is the whole point, and your setup is a good test of it.

Watch the issue Hengfei linked. Once Hermes support lands, I'd like to hear whether the per-project view gives you what you need.

@elinickerson Thanks for replying Eli, glad you're here! I think it could be helpful to share one or two anecdotes you hear from folks as they're establishing more mature AI observability practices.

@abhay_padamani1 sensitive or PII data can be handled both at ingest (which is before storage) or directly in the query layer.

Also, do you use a collector or agent? I guess we can handle some of it at source as well

@abhay_padamani1 Sensitive data Redaction handles it well for both incoming data or already ingested queryable data.
The Sensitive Data Redaction helps prevent accidental exposure of sensitive data by applying regex-based detection to values ingested into streams and to values already stored in streams.

@marketer_kim There's a built-in layer, and it runs at ingest, before anything is written — it's not left to your collector.

It's field-level: you bind a regex pattern to a stream and a field, pick a policy — drop the field, redact the match, or hash it (hashing keeps the value joinable without keeping the value) — and choose where it applies: at ingestion, at search, or both. At ingestion the value never reaches storage; at search it's stored but masked on the way out.

Prompts and responses are ordinary span fields after normalization, so the same engine that redacts your logs covers them. There's no separate LLM path to configure.

Two honest caveats: this lives in the enterprise build — on OSS today it's the collector's redaction processors. And it's regex, so it's strong on structured PII (emails, cards, keys, national IDs) and weak on names buried in free text. Anyone claiming reliable PII detection in free-form prose is overselling it.

Which do you actually need — never-stored, or stored-but-masked for most viewers? Those two pull in different directions.