If a user can't disagree with your score, it isn't a measurement
Three of today's launches ship a confident number at the user. How much of this page is AI. How far this repo sits from production ready. How many hours of debt that represents. Different products, same move.
The problem isn't accuracy. It's that nobody can tell you when it's wrong. A linter points at line 40 and you either agree or you don't. A 0-100 score for something unmeasurable can't be argued with, so it never gets corrected, and it trains people to either ignore it or trust it for no reason at all.
That's the failure I keep going on about, output that's plausible enough that nobody complains, wearing a nicer outfit.
The test I've started using before shipping a number: can a user disagree with this and be right in a way I'd have to accept? If not, it's marketing sitting inside the product. Ship the finding that points at a real thing on the screen. The score is optional and usually costs more trust than it earns.
Replies
Dial
this matches something I ran into today reading through launches - a call-scoring tool that ties its rubric to deal outcomes, which sounds rigorous until you realize a small team doesn't have enough closed-won/closed-lost data yet for that correlation to mean anything. the score looks scientific because it's tied to a real number (revenue), but if you can't ask "why did this call score 72 instead of 85" and get an answer you could actually contest, it's just a confident-looking guess wearing the outfit of a measurement, like you said. the linter comparison is the right one - a linter tells you exactly which line and why, and you can go look at it. most AI scores skip straight to the verdict and skip the part where you could catch it being wrong.
@galdayan The call-scoring case is worse than most because revenue makes it feel validated. You can be exactly wrong and still correlate, small sample plus a rubric someone wrote by feel, landing on top of a real number. The part I'd steal from linters isn't the line reference, it's that a linter can be switched off. A score a user can't disable when they think it's wrong isn't confident, it's just unfalsifiable.
Dial
@asadmalik901 unfalsifiable is the word i was missing there. most scoring tools i've seen don't even expose which inputs moved the number, so you can't argue with it, you can only ignore it. maybe the fix is smaller than a full override - just show the rubric weights behind the score, so a wrong number is at least wrong for a visible reason
@galdayan Showing the weights gets you halfway and then stops. If I see quality is 40% of the score I still can't tell you why this call got a 6 on quality, so the argument just moves down one level and dies there. What I'd want is the input, this sentence at 2:14 is why the number moved. Weights explain the formula, not the verdict.
Dial
@asadmalik901 fair, that's the harder ask though - highlighting the actual span means the model has to expose its own reasoning trace, not just its output. most of these tools weren't built to keep that around after scoring. probably why everyone stops at weights, it's the version you can ship without redoing the pipeline
Agreed. AI makes this easy now. Same in my own project: it felt fine at first, then I realized I couldn't explain where the score came from either, and I ended up writing the scoring rules out one by one.
@maxwell_wx Writing the rules out one by one is the fix, and it's also why most people avoid it. Once the rules are explicit somebody can argue with them, and arguing about a weighted list is a much less comfortable conversation than pointing at a model. Worth doing anyway. About half of ours didn't survive being written down.
@asadmalik901
The distinction that helps me is whether the number describes something now or predicts something later. A score on today's page points at a real element you can go check right away. A score on how a candidate will perform, or how a call will close, is a bet dressed up as a reading, and confirmation might never arrive. Your test holds cleanly for the first kind. The second kind adds a harder question underneath it: a user could disagree in principle and still have to wait to find out if they were right.
@alex_gidirim The predictive ones are worse because the correction arrives after you've already acted on it. If a candidate scores 62 and you skip them, you never find out you were wrong, so the number gets to stay right forever. Descriptive scores at least lose the argument the same afternoon. My rough rule now is if a number can't be proven wrong inside a week, it shouldn't be a number, it should be a sentence about what the thing actually saw.
@galdayan Shopping for the number is exactly what would have happened. Both versions sat side by side in staging for about four hours and I caught myself quoting the friendlier one in the changelog, which is the whole failure in miniature. Citation requirement stays, the second score doesn't.
@galdayan It helped, but not comfortably. The client clicked through, saw the citation was one thin paragraph, and came back saying the first number had been generous. Awkward call, still better than the version where they only find out because it moved. Thin input showing is the point. A score that hides how little it looked at is what actually burns the relationship.