SPECIMEN · Evaluation · · 1 min read
Close to Perfect, Useless in Practice
A component that scored close to perfect on its own tests and was useless against real outcomes. On the gap between what you can measure and what you can claim.
The component scored 0.98 on its own test suite. It was, by the numbers, done. Against real outcomes — the thing it was supposed to predict in the world — it was no better than a rule that always said no.
How a test suite lies
It doesn't, exactly. It answers the question it was asked. The problem is that the question drifted: the suite was built from the failures we understood, and the system was deployed against the failures we didn't.
Three ways this happens
- Leakage — the label is in the input, somewhere you didn't look.
- Selection — the test cases are the easy ones, because easy ones are the ones you can label.
- Proxy drift — the metric was a stand-in for the outcome, and the two came apart.
The through-line
This is the same problem as marketing attribution. A platform's conversion number is a proxy that scored perfectly on the platform's own test. The gap between what you can measure and what you can claim is not a bug in one field; it is the shape of measurement itself.
I find that gap more interesting than either field on its own.