Berkson paradox

A statistician at the Mayo Clinic noticed something strange in hospital data in the 1940s. Among hospitalized patients, two diseases that had no connection in the general population suddenly appeared to be negatively related, as if having one protected you from the other. There was no biological reason. The distortion came entirely from how the patients had been selected. He worked out the general principle. When you filter a group by two separate requirements at once, you create a fake negative relationship between those two traits inside the filtered group, even when they are completely unrelated in the wider world. It became known as Berkson's paradox, and it quietly infects any place where people are chosen on more than one criterion.

When you screen people on two things at once, say, smart and likeable, your selected group will look like there is a tradeoff between them, as if the smart ones are less likeable and vice versa. There usually is no such tradeoff in the real world. Your filter manufactured it, and then you mistake the artefact for a law of nature. This is worth knowing every time you feel you must choose between two good qualities in a candidate, because the sense that you cannot have both is often a shadow cast by the way you are selecting rather than reality. A close cousin, where splitting the same data reverses the answer entirely, is Simpson's paradox.

The Art of Statistics

David Spiegelhalter, 2019

A clear guide to the ways data misleads, including the traps hidden in how a sample is chosen.