What do you do when the model calls the wrong tool?
Genuine question for anyone shipping MCP servers or agents.
The model picks a tool that isn't right for the request. Or it picks the right one and fills a parameter with something you never anticipated. What's your actual process for fixing that?
Options I've seen people reach for:
- Rewrite the description and hope. Cheap, but you're guessing at what confused it and you can't tell whether the rewrite helped.
- Add logging in the handler. Works, but only shows what happened after the model chose. You never see the tool it nearly picked instead.
- Merge tools into one with a mode parameter. Less to choose between, at the cost of a fatter interface.
- Split the server. Fewer tools per connection, more servers to maintain.
- Reproduce it manually and iterate. Slow, and your test prompts aren't what real users type.
What I want to know is whether anyone has a repeatable method here, or whether it's guesswork for all of us. Every developer I've asked describes some version of change the wording, restart, see if it feels better.
One thing that pushed me toward looking at schemas: I read the tool definitions of 4,951 public MCP servers, and 17.4% of tools contain no word distinguishing them from a sibling tool on the same server. In those cases the model has nothing in the descriptions to go on — the tool names are doing all the work.
But that's static analysis. It can't tell me how often it actually costs anything, which is why I'm asking people who've watched it happen.
So: what's worked for you, and what turned out to be a waste of time?
Replies