The pervasiveness of the belief in the hot hand in basketball is widely accepted. There is less agreement regarding the reality of this phenomenon in the basketball court. It is here that null hypothesis significance testing (NHST) can make a contribution. Rather than testing whether a random model perfectly predicts performance, one can test whether there is a significant directional effect. The null hypothesis is thus not a meaningless point hypothesis, but the upper bound of half the available continuum (which represents the cold hand).
2. The presumed phenomenon of the hot hand in basketball offers a unique opportunity for the study of cognitive bias because it involves statistical analysis at both the level of human perception and at the level of the corresponding behavioral reality. Gilovich et al. present a convincing case that most people, athletes and spectators alike, are firmly convinced of the reality of the hot hand. They believe that for most players, the conditional probability of making a shot is greater after one or several shots have succeeded than after one or several shots have failed. In statistical terms, this conviction is so strong and pervasive that inferential tests add little information. That is, the alternative conviction that there is a cold hand in basketball is not a serious contender. What makes the study by Gilovich et al. compelling is the examination of the behavioral reality. They ask whether the conditional probability of success after success is actually greater than the conditional probability of success after failure. If it were true that the hot hand existed, the belief in the hot hand would seem less irrational even if it were exaggerated in magnitude.
3. Thus, the study of the hot hand is a case in which support of the null hypothesis supports a claim regarding irrational perception. People seem misguided if they perceive systematic patterns of success and failure when in fact success at shooting baskets is well predicted by a random model. The question becomes whether NHST can adequately support the random model of reality in this domain. Gilovich et al. argue that it can, whereas Hallahan suggests that nonrandomness can be detected in actual sequences of shots. If nonrandomness exists, perceptions of nonrandomness may not be wrong.
4. I am less concerned with the limitations of NHST in this particular example than Hallahan is. My own concerns are focused on domains in which perceptual biases are detected on either side of the null distribution. The examples I used in the target article included the co-existence of opposite biases in the literature on social perception (false consensus and false uniqueness, self-enhancement and self-diminishment, overattribution and underattribution). In these examples, rationality is equated with the absence of any bias, be it a positive or a negative one. Misperception of chance in the game of basketball, however, is only of one kind. There is no hypothesis and no evidence that people might expect a cold hand. This is a crucial observation because it allows the null hypothesis to be directional instead of point specific.
5. Suppose for a moment that the null hypothesis of success at basket shooting is point specific. In this case, the expected serial correlation between success and failure at successive shots is exactly zero. Because a point specific hypothesis has no probability, a sufficiently large sample of players and shots would ultimately result in a significant positive or negative deviation from chance. Because only a hot hand but not a cold hand is expected, however, the null hypothesis (zero correlation) can be understood as the upper bound of the negative range (Dawes 1997). As long as there is no significant positive correlation in reality, people who believe in such a correlation are biased. Indeed, the data Hallahan reviews suggest that by and large basketball players display a "cool hand." The conditional probability of making a shot after a success is somewhat lower than the conditional probability of making a shot after a failure. Combining data meta-analytically, Hallahan finds these trends to be significant. It seems, therefore, that the belief in the hot hand is truly mistaken.
6. A final comment is in order on Hallahan's suggestion that the serial correlations in the 1998 data set are not normally distributed. The data consist of 8 sequences of 25 shots each. Most (5) of the serial correlations are negative and the overall mean is -.02. Hallahan notes that only 1 correlation falls within the range between -.055 and .184 as opposed to 3 such correlations that one would expect if the distribution were normal and centered around zero. Further inspection of the data shows that the actually obtained distribution is spread out and positively skewed. For example, the mean negative correlation is -.22, whereas the mean positive correlation is .30. It thus seems possible, as Hallahan suggests, that there are systematic individual differences in streak shooting that are obscured by the typical group-focused NHST procedures. The standard NHST approach accepts the random model of streak shooting in toto, assuming that both -- sequences within players and differences between players -- are random. Future research would benefit from a separation of these two sources of variance. Indeed, Gilovich et al. (1985) opened their article with the observation that spectators selectively attribute streak shooting to the best players rather than to the average player or all players [2].
[1] These data can be found at http://members.xoom.com/allstar98/3ptsshoutout_round.html
[2] Of course, the players who spectators believe to be streak shooters may not be the ones who actually do have a hot hand.
Dawes, R. M. (1997) Qualitative consistency masquerading as quantitative fit. In M. L. Dalla Chiara et al. (Eds.), Structures and norms in science (pp. 387-394). Kluwer Academic Publishers.
Gilovich, T., Vallone, R. & Tversky, A. (1985). The hot hand in basketball: On the misperception of random sequences. Cognitive Psychology, 17, 295-314.
Hallahan, M. (1998). The hazards of mechanical hypothesis testing. PSYCOLOQUY 10(001) ftp://ftp.princeton.edu/pub/harnad/Psycoloquy/1999.volume.10/ psyc.98.9.001.social-bias.13.hallahan http://www.cogsci.soton.ac.uk/cgi/psyc/newpsy?10.001
Krueger, J. (1998). The bet on bias: A foregone conclusion? PSYCOLOQUY 9(46) ftp://ftp.princeton.edu/pub/harnad/Psycoloquy/1998.volume.9/ psyc.98.9.46.social-bias.1.krueger http://www.cogsci.soton.ac.uk/cgi/psyc/newpsy?9.46