Stop measuring AI agents by "containment rate"
by•
Containment rate (the share of conversations the bot handles without a human) is the default AI support metric. We think it rewards the wrong thing. A bot can contain a conversation by never letting the customer reach a person.
With customers, we track the outcome instead: was the payment made, the policy renewed, the meeting booked, the issue resolved? One real estate customer cut outreach costs by about 70% and finished a month of calls in under a tenth of the time. That's a number a CFO cares about. Containment isn't.
What metric does your team use to decide if an AI tool is worth keeping?
Support our launch, last 1 hour remaining!! https://www.producthunt.com/products/unleashx
29 views
Replies
Agree, and back-office work has the same trap. We look at cost per completed task, where completed means it’s actually done and nobody had to redo it a week later. An agent that handles 90% of items but quietly gets 5% wrong can cost more than one that honestly hands off 30%. I’d track rework rate next to whatever outcome metric you pick. That number tells you whether the handoffs are happening in the right places.