Krueger is right to call for more flexible statistical methods when evaluating cognitive biases and for a consideration of individual differences. A programmatic series of studies on individual differences in rational thought (Stanovich, in press) illustrates how patterns of covariance among reasoning biases -- as well as patterns of covariance among biases and indices of cognitive ability -- can help to reveal when discrepancies between normative and descriptive models are due to performance errors, to computational limitations, and to the misapplication of normative models by experimenters. Patterns of individual differences have implications for alternative explanations of the gap between normative and descriptive models of human behavior.
2. Our analyses are similar to those Krueger (1998b) describes for a social projection experiment in his reply to Hamm (1998), in that we have tried to examine "whether degrees of rationality may predict other variables of interest" (Krueger, 1998b, paragraph 2). In particular, we have examined whether individual differences in cognitive biases are intercorrelated and whether quantitative measures of these biases correlate with indirect indicators of computational capacity (i.e., measures of intelligence or cognitive ability).
3. Stanovich (in press) describes how such analyses have implications for alternative explanations of the gap between normative and descriptive models of human behavior. For example, in various critiques, investigators have argued that: experimenters have observed performance errors rather than systematically irrational responses; that the tasks have required computational operations that exceed human cognitive capacity; that experimenters have applied the wrong normative model to the task; and that participants have misinterpreted the tasks. Stanovich demonstrates how patterns of intercorrelation among individual differences can throw light on these alternative explanations. For example, in certain cases, negative correlations between a bias and cognitive ability can indicate that the normative model is not prescriptive for some subjects (due to computational limitations). But at the same time, such a correlation can serve to validate the experimenter's choice of normative model with which to evaluate performance. Such a pattern was apparent in our analyses of the four-card task, the belief-bias effect in syllogistic reasoning, the conjunction fallacy, outcome biases, inductive reasoning tasks of the type studied by the Nisbett group, my-side bias in informal reasoning, and several other tasks.
4. However, individual difference patterns were observed on some tasks that called into question the normative model being used to evaluate performance. Interestingly, given Krueger's target article, the social projection experiment was one such task. We found that individuals who tended to project their own opinions did not differ from those who projected less in their level of cognitive ability or in their responses on other reasoning tasks. These findings are consistent with the arguments of Krueger and Zeiger (1993), Dawes (1989), and Hoch (1987) that projecting consensus is not necessarily inefficacious and thus should not be considered nonnormative. Individuals high in projection tendency were in fact somewhat more accurate in their opinion estimates in our experiments -- and they were not more prone to other cognitive biases, nor were they low in cognitive ability or other rational thinking dispositions that we measured with questionnaire instruments. Although the consensus effect has traditionally been interpreted as an egoistic bias that was unjustified from a normative point of view, there was little evidence for such an interpretation in the individual-differences analyses that we have conducted.
5. Parallel individual-differences analyses on another task studied by Sa, West, and Stanovich (1998) revealed trends supportive of the normative critique which Krueger (1998a) makes of the projection of consensus in the social projection experiment: "In consensus estimation, the criterion of no difference is the wrong benchmark for rational predictions. People's own responses are by definition more likely to be the responses of the majority than the responses of the minority. Therefore, one's own responses should be used for consensus estimation to minimize errors. This strategy leads to results that look like consensus bias, although it is based on normative inductive reasoning" (paragraph 15). In the Sa et al. (1998) study, what was studied was not subjects' projections of their own opinions, but their knowledge of the structural characteristics of the world and their ability to project this knowledge in appropriate and inappropriate circumstances. The task was adapted from the work of Nelson, Biernat, and Manis (1990), who had participants judge the heights of photographs of seated and standing males and females. Of course, gender is a valid cue when the photographs to be judged are representative of the population. This was the case in the so-called ecological condition of the Sa et al. (1998) investigation. However, the gender cue was rendered nondiagnostic in a so-called matched condition where the male and female photographs were equated in height (and participants were told of the matching).
6. Measures of cognitive ability were positively correlated with gender projection in the ecological condition where prior knowledge accurately reflected an aspect of the perceptual environment. But when the perceptual judgment task involved prior knowledge which was incongruent with the perceptual environment (the matched set), projection of the gender cue was negatively associated with cognitive ability. Thus, people with more intellectual resources appear to be able to use prior knowledge flexibly depending upon its efficacy in a particular environment. They are more likely to project a relationship when it reflects a useful cue, but they are also more likely to decouple prior belief from their judgments when it is inefficacious. More intelligent individuals do contextualize the problem more when that context contains cues that can facilitate judgment; but they are less likely to carry over contextual cues into situations where they know that the cues are no longer diagnostic. Projection in this task is negatively correlated with cognitive ability only when it lacks normative justification.
7. These and similar analyses (see Stanovich, in press) illustrate how patterns of covariance among reasoning biases -- as well as patterns of covariance among biases and indices of cognitive ability -- can help to reveal when discrepancies between normative and descriptive models are due to performance errors, to computational limitations, and to the misapplication of normative models by experimenters. Our work on individual differences reinforces Krueger's (1998a) call for more flexible statistical models to be used in the assessment of reasoning biases -- but we have worked to establish an even more fundamental point. We have simply asked why both proponents of the heuristics and biases approach (and equally their critics) have focused entirely on the central tendency of responses (usually the mean or modal performance tendency). We have tried to focus the field instead on the variability in responses and the rich patterns of covariance present in multivariate experiments, and to show that these patterns have implications for alternative explanations of the gap between normative and descriptive models of human behavior.
8. Finally, on the exchange between Krueger and Rickert (1998) on intelligence and rationality, my perspective is slightly different from that of either of these investigators. In Stanovich (in press) I use common distinctions among levels of analysis in cognitive theory (Anderson, 1990; Dennett, 1987; Marr, 1982; Newell, 1982, 1990; Oaksford & Chater, 1995), identifying intelligence with computational capacity at the algorithmic level of analysis and with considerations of rationality at the intentional level of analysis. This partitioning provides a useful framework for empirical investigations of individual differences in rational thought.
Anderson, J. R. (1990). The adaptive character of thought. Hillsdale, NJ: Erlbaum.
Dawes, R. M. (1989). Statistical criteria for a truly false consensus effect. Journal of Experimental Social Psychology 25: 1-17.
Dennett, D. C. (1987). The intentional Stance. Cambridge, MA: MIT Press.
Hamm, R. M. (1998) Characterizing individual strategies illuminates nonoptimal behavior. PSYCOLOQUY 9(49) http://www.cogsci.soton.ac.uk/cgi/psyc/newpsy?9.49 ftp://ftp.princeton.edu/pub/harnad/Psycoloquy/1998.volume.9/psyc.98.9.49.social-bias.2.hamm
Hoch, S. J. (1987). Perceived consensus and predictive accuracy: The pros and cons of projection. Journal of Personality and Social Psychology, 53: 221-234.
Krueger, J. (1998a). The bet on bias: A foregone conclusion? PSYCOLOQUY 9(46) http://www.cogsci.soton.ac.uk/cgi/psyc/newpsy?9.46 ftp://ftp.princeton.edu/pub/harnad/Psycoloquy/1998.volume.9/psyc.98.9.46.social-bias.1.krueger
Krueger, J. (1998b). Getting to the core of the data by testing against alternative hypotheses. PSYCOLOQUY 9(70) http://www.cogsci.soton.ac.uk/cgi/psyc/newpsy?9.70 ftp://ftp.princeton.edu/pub/harnad/Psycoloquy/1998.volume.9/psyc.98.9.70.social-bias.8.krueger
Krueger, J., & Zeigler, J. S. (1993). Social categorization and the truly false consensus effect. Journal of Personality and Social Psychology 65: 670-680.
Marr, D. (1982). Vision. San Francisco: W. H. Freeman.
Nelson, T., Biernat, M., & Manis, M. (1990). Everyday base rates (sex stereotypes): Potent and resilient. Journal of Personality and Social Psychology, 59, 664-675.
Newell, A. (1982). The knowledge level. Artificial Intelligence, 18, 87-127. Newell, A. (1990). Unified theories of cognition. Cambridge, MA: Harvard University Press.
Oaksford, M., & Chater, N. (1995). Theories of reasoning and the computational explanation of everyday inference. Thinking and Reasoning, 1: 121-152.
Rickert, N. W. (1998a) Intelligence is not rational. PSYCOLOQUY 9(51) http://www.cogsci.soton.ac.uk/cgi/psyc/newpsy?9.51 ftp://ftp.princeton.edu/pub/harnad/Psycoloquy/1998.volume.9/psyc.98.9.51.social-bias.3.rickert
Ruscio, J. (1998) Applying what we have learned: Understanding and correcting biased judgment. PSYCOLOQUY 9(69) http://www.cogsci.soton.ac.uk/cgi/psyc/newpsy?9.69 ftp://ftp.princeton.edu/pub/harnad/Psycoloquy/1998.volume.9/psyc.98.9.69.social-bias.7.krueger
S, W., West, R. F., & Stanovich, K. E. (1998). The domain specificity and generality of belief bias in reasoning and judgment. Manuscript under review.
Stanovich, K. E. (in press). Who is rational? Studies of individual differences in reasoning. Mahweh, NJ: Erlbaum.
Stanovich, K. E., & West, R. F. (1998a). Cognitive ability and variation in selection task performance. Thinking and Reasoning, 4, 193-230.
Stanovich, K. E., & West, R. F. (1998b). Individual differences in rational thought. Journal of Experimental Psychology: General, 127, 161-188.
Stanovich, K. E., & West, R. F. (in press). Individual differences in framing and conjunction effects. Thinking and Reasoning.