This one gets repeated everywhere. When researchers went back and checked the original evidence, it did not hold up. The popular version is not what the data actually shows.
It is one of the most repeated findings in management: workers at the Hawthorne factory near Chicago, studied in the 1920s and 1930s, produced more no matter what the researchers changed. Brighten the lights, output rose. Dim the lights, output rose. Change the breaks, the hours, almost anything, and productivity went up, apparently just because the workers knew they were being watched and studied. The Hawthorne effect, the idea that being observed makes people perform better, entered every textbook and shaped decades of workplace thinking. Then, around eighty years later, economists tracked down the original data, which many had assumed was lost, and re-analysed it properly. The famous pattern was not there. Much of the supposed effect was ordinary things dressed up, output tended to rise on Mondays regardless of any change, and improved over time as workers gained experience, whether or not anything was being altered. The dramatic being-watched effect that everyone cites turned out to be far weaker, and largely an artefact of how the original story was told.
A finding can be taught in every textbook for eighty years and still not be supported by its own original data, once someone finally goes back and checks. The Hawthorne effect felt intuitive and told a satisfying story, so it survived for generations largely unexamined. Sometimes being observed really does change behaviour. The specific famous claim, though, was much shakier than its fame suggested. The wider lesson is to hold even the most established, universally repeated findings with a little suspicion, and to notice that repetition is not evidence. Something being said everywhere, for a very long time, is not the same as it being true, and occasionally nobody checks until decades too late.
Source: Levitt and List, 2011, American Economic Journal, re-analysing the original data. Review: McCambridge and colleagues, 2014.
Jim Manzi, 2012
On why rigorous testing matters, and how much accepted wisdom does not survive an honest re-examination.
Draw your own card. It does not take long, and it rewards taking your time.