← Back to the library

    Normalization of deviance

    holds up

    This is documented history rather than a laboratory result. The events are well recorded. What they teach is an argument, and the entry makes it openly.

    On 28 January 1986 the space shuttle Challenger broke apart 73 seconds after launch, in conditions far colder than any shuttle had flown in. Engineers at the company that built the rocket boosters had formally recommended against launching below 53 degrees Fahrenheit. They were overruled.

    The obvious explanation was negligence. The sociologist Diane Vaughan spent nine years inside the documents and arrived somewhere else. The rubber seals between the booster segments, the parts that failed, had a simple design requirement: they should come back undamaged. From the second shuttle flight onward, they did not. Erosion appeared, then appeared again, and each time the vehicle returned safely, so the damage was reviewed, explained and formally classified as an acceptable risk. The next instance was then measured against the last one rather than against the original requirement. Over nine years and two dozen flights, the acceptable amount of damage kept moving, one small revision at a time, each one defensible on the evidence available.

    Vaughan's phrase for what happened is that they redefined evidence that deviated from an acceptable standard so that it became the standard. Nobody broke a rule. The people making these judgments were competent, and the flights kept coming home, which is precisely what made the drift invisible. By January 1986 the argument on the table was not whether damaged seals were tolerable, since that had been settled years earlier. It was whether the cold made this particular flight different, and the engineers who thought so could not prove it.

    Getting away with something feels identical, from the inside, to the thing being safe. That is the whole mechanism, and it is why capable people in good faith are the ones it happens to.

    The pattern is easy to find once you know its shape. A standard exists. Something falls short of it. Nothing bad happens, so the shortfall gets explained rather than corrected, and the next shortfall is compared to the last one instead of to the standard. The overloaded engineer who has not burned out yet. The review that gets skipped when the deadline is tight, and then gets skipped again. The check that everybody knows is theatre. None of these is a decision to accept more risk. Each one is a small reclassification of what normal means.

    Two things make it worse in a company than at NASA. Nobody writes the standard down, so there is no original to drift from, only a memory of how things used to be done. And the evidence keeps arriving on the wrong side, because every uneventful month is read as proof that the shortcut is fine.

    The question that finds it takes one pass through your own operation: what is being tolerated now that would have alarmed you a year ago? If the answer arrives quickly, the drift has already happened, and the reason it is not frightening is that nothing has gone wrong yet.

    Source: Diane Vaughan, The Challenger Launch Decision: Risky Technology, Culture, and Deviance at NASA, University of Chicago Press, 1996, built on nine years of archival research and interviews. The concept has since been adopted in aviation and healthcare safety, including by NASA's own safety organisation. Richard Feynman's appendix to the 1986 Presidential Commission report, Personal Observations on the Reliability of the Shuttle, reaches a compatible conclusion from the engineering side.

    The book, if you want to go further

    The Challenger Launch Decision

    Diane Vaughan, 1996

    Nine years of records, reconstructed decision by decision. It is a demanding read and the argument is worth it: no villains, no broken rules, and a disaster that assembled itself out of reasonable judgments.

    Draw your own card. It does not take long, and it rewards taking your time.