Regression to the mean

Galton plotted the average height of both parents against the height of their adult children and found a level of mediocrity at about 68.25 inches. Above it, children came out shorter than their parents. Below it, taller. He called the pattern regression towards mediocrity and read it as heredity pulling a species back toward its type. Later statisticians showed no pull is needed. The same pattern appears between any two things that are imperfectly correlated, and it runs in both directions, which a hereditary force could not do.

In 1933 a professor at Northwestern named Horace Secrist published the business version of Galton's mistake, at book length. The Triumph of Mediocrity in Business ran to more than two hundred charts and tables and had taken ten years. He had sorted department stores, retail clothing shops and national banks by their 1920 performance and followed them to 1930. The firms that started at the top finished only slightly above average. The firms that started at the bottom finished only slightly below it. Secrist concluded that competitive pressure grinds every business toward mediocrity, and suggested it might lie behind the Depression.

Harold Hotelling reviewed the book in the Journal of the American Statistical Association the same year. The convergence, he showed, was an artefact of how the firms had been grouped, and the results proved nothing except that the ratios wander. The data was accurate. The conclusion was arithmetic wearing an economic costume. Secrist replied, and Hotelling answered again the following year.

The most useful demonstration came decades later. Daniel Kahneman was teaching a group of military flight instructors that rewarding good performance works better than punishing bad. An experienced instructor objected from his own record: praise a cadet for an excellent manoeuvre and the next one is worse, shout at him for a poor one and the next one is better, so criticism works and praise backfires. Both patterns were regression. An unusually good manoeuvre is followed by a more ordinary one whether anyone says anything at all.

People notice the pattern. The practical danger is that it hands you evidence for whatever you happened to do.

You act at the extreme, because the extreme is what gets your attention. The consultant is called in after the worst quarter. The manager is replaced after the bad year. The performance plan lands on the weakest performer. The agency is fired after the campaign that missed. Then things improve, and the improvement was coming anyway, and you have just learned something untrue about what works.

It runs the other way too, and that version is quieter. Your best month is usually followed by a worse one. The hire who interviewed brilliantly usually performs like everybody else, because an interview is a single measurement and a spectacular one contains luck. Neither of those needs an explanation, and both will get one.

The nastiest version sits inside management. A manager who only intervenes at the extremes will accumulate a career's worth of evidence that criticism works and encouragement does not, for the same reason the flight instructor did. The evidence is real. The lesson is wrong.

The question that defuses it is compared to what. There are two ways to answer it.

The first is a comparison group, something you deliberately left alone while you changed the other thing. Two ad sets, one rebuilt and one untouched. Two regions, one that got the new process and one that did not. Whatever the untouched one does is the recovery you were going to get anyway, and only the gap between them belongs to your decision.

The second works when you have nothing to compare against, which is most of the time in a small business. Go back through your own numbers and find out how your bad stretches have ended before. If the last four dips recovered inside two months without anyone doing anything, then a dip that recovers inside two months after you act tells you nothing. A dip that recovers in three weeks might. You are comparing against your own history rather than against a control. It is weaker, and it is a real question with a real answer.

A before and after taken at an extreme tells you almost nothing on its own, and it will feel like it tells you everything.

Source: Galton, Regression towards Mediocrity in Hereditary Stature, Journal of the Anthropological Institute of Great Britain and Ireland, 1886. The business blunder and its correction: Secrist, The Triumph of Mediocrity in Business, Northwestern University Press, 1933, and Hotelling's review, Journal of the American Statistical Association, 1933, volume 28, pages 463 to 465. The flight instructors are in Kahneman, Thinking, Fast and Slow, 2011, chapter 17.

Thinking, Fast and Slow

Daniel Kahneman, 2011

Chapter 17 is called Regression to the Mean, and the flight instructor story is Kahneman's own. The pattern stayed hidden from a room of experienced instructors who had a career of real observations pointing the other way, which tells you how hard it is to see from inside your own evidence.