The Fight, Episode 1: How to Tell Which of Your Metrics Are Lying

I made this because I kept watching the same argument in different meetings. The measurement people quote Moneyball at you, the judgement people quote Vietnam at you, and both of them have real evidence and have won real arguments. Most advice picks one of them and pretends the other side's cases never existed, and that is what I wanted to stop doing.

I am not neutral on it, and the episode says so early. Some years ago I hired an agency for my e-commerce brand, got angry about the numbers, and told them exactly which metrics I wanted to see move. They delivered those metrics with cheap traffic that did not sell anything, which is a bounty in its purest form, and I was the one who wrote it. Then in 2025, with close to twenty years of hiring behind me, I interviewed for a CTO by talking more than listening, liked the candidates who repeated my own priorities back to me, and made three expensive hires in a row. One loss was to a number that got gamed and the other was to my own instinct, and neither was anybody else's fault.

Where I have landed is the stand at the end of the episode. In any decision I make repeatedly, hiring, pricing, which campaign gets the budget, I trust structure over my own hunch, because that is where the evidence sits. And I stay suspicious of any decision that rejects something only because the numbers did not support it, because numbers only ever contain what somebody managed to count.

The opening, McNamara and Oakland. In 1961 Robert McNamara brought one of the most precise reporting systems any war has had to the Pentagon, and it was right about everything it counted and wrong about the war. In 2002 Billy Beane stopped listening to his scouts, trusted a spreadsheet, and won twenty games in a row on about a third of the Yankees' payroll. Same method, opposite outcomes, and that is the problem the episode exists to solve. The two cases are the McNamara fallacy and the good face.

The case against the numbers. A reward turns a number into a target, and a target stops describing the thing it was meant to measure. The cleanest documented case is Hanoi in 1902, where a bounty on rat tails produced rat farms, and the famous cobra version of the same story has no solid primary record. That is Goodhart's law. Rory Sutherland's argument sits on the same side: a spreadsheet can only hold what is easy to count, and some of what wins markets is very hard to count, which he tells through Red Bull failing every test it was put through. That one is graded debated.

The case for the numbers. The scouts' trained eye decided careers for a hundred years and was measuring very little, and on-base percentage, which nobody was paid to move, predicted scoring better. The same finding turns up somewhere much messier, in hiring: eighty-five years of research pooled into one ranking puts the friendly unstructured interview well below the structured methods, and a 2022 revision widened that gap. See what actually predicts job performance.

The thermometer. A thermometer tells you the truth right up until somebody is paid a bonus for a low reading, and then it can be held under a cold tap. Same instrument, same numbers on the glass, and the reading now describes the tap. Numbers with somebody's money on them are bounties, numbers with nobody's money on them are diagnostics, and on a dashboard the two look identical.

The question, and two checks. The question the episode promises is whether anybody's reward depends on the number. The first check is action: can a named person change the number, and what would they do on Monday to change it. The second is for the meeting where someone says the numbers are missing the point, which is sometimes Red Bull and sometimes the scouts: make them name what the number cannot see, then ask what has actually happened because of it.