Makers who've shipped AI agent systems in production, what broke that you didn't expect?

Building production agent systems is genuinely hard in ways that demo videos don't show.

The failure modes I keep running into:

• Cascading quality degradation when agent confidence doesn't propagate between transitions

• Race conditions in the shared state under concurrent load

• Silent errors where the orchestration reports success, but the final output is quietly wrong

That last one is the most dangerous. Everything looks fine until a real user hits it.

Genuinely curious what others have experienced:

• What broke between demo and production that you didn't anticipate?

• How are you handling observability, full trace visibility, or debugging blind from just the final output?

• What's your human-in-the-loop escalation approach, threshold-based or manual?

Not looking to sell anything. Just trying to figure out whether these patterns are universal or architecture-specific. Feels like this space is moving too fast for anyone to have clean answers yet.