What do you do when the feature everyone asked for is the one that makes the product worse?

For months the single most requested thing was a chat style interface. Users said they wanted to talk to the product instead of clicking through it. We built it, shipped it and watched core metrics slide within two weeks. The people who asked for it used it a few times and drifted. The format had changed how they related to the tool from reflecting on what they were doing to just asking for an answer. We pulled it and replaced it with something guided with pauses built in. That version worked better than both. But we only found it because we questioned why the request existed, not just what it was. How do you separate "users want this" from "users will actually use this" before you build it?