I’m going to resist summarizing Miller & Sanjurjo’s “Surprised by the Gambler's and Hot Hand Fallacies? A Truth in the Law of Small Numbers,” not only because I’ve already tried to do that multiple times –
- Holy smokes! The " 'hot-hand fallacy' fallacy"!
- Still fooled by non-randomness? Some gadgets to help you *see* the " 'hot hand fallacy' fallacy
but also because any attempt to do so results in a mental misadventure of staggering proportions.
Actually, that is what’s so cool about the article. At least in my view.
Like lots of other people—including, to his credit, the scholar most prominently identified with the classic “hot hand fallacy” study—I think it is really neat that M&S have re-opened the question whether the performance of athletes really do vary in patterns that defy the fluctuations one would expect to see by chance (i.e., whether NBA basketball players and others really do go on “hot streaks” etc).
I also am filled with admiration for their mathematical dexterity in exposing the error in the original “hot hand fallacy” research (viz., the assumption that the shooting consistency of basketball players over a finite set of observations should be measured in relation to the variance associated with a binomial distribution).
But what really intrigues me is what M&S's accomplishment tells us about cognition. Or really what it tells us about what we don’t know but should about how intuition and conscious reflection operate in expert judgment.
How could researchers so familiar with probability theory, and so accomplished in exposing the errors people routinely make when attempting to detect patterns in random events, fail to detect the mistaken assumption that they themselves were making about how to detect such a pattern in this particular setting?
How could the error have evaded the notice of those who reviewed their work—and much more fundamentally the notice of thousands of scholars who for decades have held up the original “hot hand fallacy” study (along with its many progeny) as the paradigmatic demonstration of a particular cognitive bias (one that no one disputes really exists) and of a method for detecting defects in human rationality generally?
Why when they are shown incontrovertible (really!) proof of the error that the “hot hand” researchers made (and re-made over the course of numerous successor studies) do so many highly intelligent, reflective people—ones who unquestionably possess the knowledge and reasoning proficiency that it takes to understand the logic of the M&S argument—so strongly and stubbornly resist accepting it before (in the vast majority of cases, at least) finally acknowledging (often with a gratifying display of appreciative surprise) that M&S are right?
What is the cognitive process, in short, that makes individuals who have cultivated the habits of mind necessary to resist commonplace but mistaken intuitions about randomness vulnerable to being misled by mistaken intuitions about randomness that only those highly proficient in reasoning about randomness could have developed in the first place?
The project to answer this question started before 2015.
But the vividness imparted to this puzzle by the astonishing M&S paper, and the resulting amplification and dissemination of the motivation to solve it, will, I predict, energize researchers for years to come.