Gary L. Wells and Paul D. Windschitl (1994) When is the use of Base-rate Information not a Logical Imperative?. Psycoloquy: 5(33) Base Rate (14)

Volume: 5 (next, prev) Issue: 33 (next, prev) Article: 14 (next prev first) Alternate versions: ASCII Summary
Topic:
Article:
PSYCOLOQUY (ISSN 1055-0143) is sponsored by the American Psychological Association (APA).
Psycoloquy 5(33): When is the use of Base-rate Information not a Logical Imperative?

WHEN IS THE USE OF BASE-RATE INFORMATION NOT A LOGICAL IMPERATIVE?
Commentary on Koehler on Base-rates

Gary L. Wells and Paul D. Windschitl
Iowa State University
Department of Psychology
Ames, Iowa 50011

S1.GLW@ISUMVS.IASTATE.EDU S1.PDW@ISUMVS.IASTATE.EDU

Abstract

The view that people fail to use base-rate information appropriately in making various judgments and decisions may have a weaker empirical and logical foundation than many judgment and decision scientists have assumed. Koehler (1993) describes several problems with the base-rate neglect thesis. We agree with Koehler and we offer two examples, one from the attribution literature and one from a legal decision context, to argue that there is no logical imperative mandating the way base-rate information should be used for these types of judgments and decisions.

Keywords

Base rate fallacy, Bayes' theorem, decision making, ecological validity, ethics, fallacy, judgment, probability.

I. INTRODUCTION

1. People engage in many behaviors that are dysfunctional in one sense or another. The list of possible examples is almost endless, ranging from small financial matters such as not consolidating credit card debt to more life-threatening behaviors such as drunk driving or risky sexual practices. Cognitive psychologists have been particularly concerned with the processes that give rise to faulty judgment rather than dysfunctional behaviors themselves, but it is presumed that these judgments mediate bad decisions and resultant behaviors. The observation that people's subjective probabilities commonly diverge from the "actual" probabilities, as defined by normatively prescriptive rules such as Bayes' Theorem, has led to a large literature on the way people process probability information to reach various judgments (e.g., see Kahneman, Slovic & Tversky, 1982).

2. A general conclusion permeating the empirical and theoretical literature is that people tend to ignore or underutilize base-rate information when making judgments and decisions (e.g., Bar-Hillel, 1980, 1983; Borgida & Brekke, 1981; Christiansen-Szalanski & Bushyhead, 1981; Nisbett & Borgida, 1975; Pollard & Evans, 1983). Koehler (1993) has called this general conclusion into question on several grounds. Among the bases of Koehler's reluctance to accept the base-rate neglect conclusion are: (a) the empirical literature does not support a general conclusion that people ignore base rates, (b) few real-world tasks map directly into the Bayesian framework that is held as the standard of good decision making, and (c) it is unclear how real-world base rates should be used by decision makers.

3. We agree with Koehler's (1993) general argument and most of the specific points he raises. In particular, we are in agreement with the argument that base-rate information may not map into certain types of problems in a manner that is required by normative considerations of Bayes' Theorem or any other logical or mathematical imperative. We offer two examples of our point. One derives from the attribution literature in social psychology, where it has been argued that consensus information (about the proportion of people who behave in a particular manner in a given situation) is a form of base-rate information that should be used to make judgments about the locus of the cause of an individual person's behavior. People's tendency not to use consensus information in the specified way has been interpreted as evidence for base-rate neglect. We argue that this interpretation is not warranted. Our second example is from the psycho-legal literature where there is often a rejection of base-rate information in making judgments of legal liability. There are sound reasons for rejecting base rates as forms of evidence for making decisions of this sort.

4. The common element in our examples is that the judgment or decision required of the subject is something other than a probability estimate. In general, we argue that studies using dependent measures that are not subjective probabilities are often riddled with assumptions about normative use of base rates that do not follow from strict logical or mathematical considerations.

II. ATTRIBUTIONS AND BASE-RATE REASONING

5. Most of the empirical literature on base rate utilization has used people's estimates of probability or likelihood as the dependent measure under various manipulations of base-rate information, but it is also common for base-rate theorists to draw on research that uses other dependent measures, such as attribution literature in social psychology, as "additional evidence" that people neglect base-rate information. The apparent parallelism between ignoring consensus information in making attributions and ignoring base-rate information in probability estimates was first noted by Nisbett and his colleagues (Nisbett & Borgida, 1975; Nisbett, Borgida, Crandall & Reed, 1976). The basic argument is that people should shift their attributions about a target person's behavior toward external causes (e.g., situation, environment, circumstance) and away from internal causes (e.g., personality, traits or other dispositions) to the extent that the behavior is perceived as common (i.e., having a high base rate) for people in that situation. Hence, for example, according to attribution theory, people should attribute a bystander's failure to provide aid to a person in need to external causes to the extent that they learn that other bystanders also do not come to the aid of a needy person under similar conditions. When the population base rate for the target person's behavior is high, internal attributions for the target person's behavior (e.g., traits such as unhelpful or other underlying dispositions) should not be assumed to be the cause of the target's behavior. To some extent, people do shift their attributions as a function of manipulations of the behavioral base-rate information (e.g., see Wells & Harvey, 1977; 1978), but one cannot help but be impressed by the relatively conservative magnitude of these attributional shifts and the lengths to which one must go at times to get people to use such information in making attributions.

6. Although research on the underutilization of consensus information in attribution has been fruitful in helping develop a better understanding of attribution processes in human judgment, we seriously question the premise that people ought necessarily to use base-rate information in the particular way that attribution researchers have described as "normative" or "correct." Although we would probably agree that people should consider base-rate information about the behavior of a sample of bystanders if their task is to predict an individual's likelihood of helping, we know of no mathematical necessity for incorporating this base-rate information into their attributions about a target person's behavior. Suppose, for example, subjects were told that a particular person did not come to the aid of someone who needed help. Suppose further that half of the subjects were told that 75% of people in general do not aid a person under these conditions whereas the other half of the subjects are told that only 25% of people fail to come to the aid of a person under these conditions. Is it irrational for subjects to make equally strong internal attributions (e.g., ratings of unhelpfulness) for the target person's behavior under these two base-rate conditions? Not necessarily. Subjects could use the base-rate information in a different manner by revising their assumptions about the prevalence of the trait (e.g., unhelpfulness) in the general population. In other words, when the attributor learns that 75% of people in general behave the same way that the target behaved, the attributor could assume that there must be many unhelpful people in the population. We see no logical necessity for attributors to revise their attributions about the target's behavior.

7. In the case of attributions, researchers need to be cautious in assuming that normative rules for probabilistic information can be translated into a rational imperative that mandates a particular attributional response. The suggestion that manipulations of consensus (behavioral base rate) information should affect attributions is not a mathematical or logical argument but a theoretical assumption about how people might make attributions. For example, as the public learns that the base rate for child abuse is much higher than previously thought, there is no rational requirement that attributions about abusers must shift toward external causes. It might be equally rational for people merely to assume that the internal dispositions producing child abuse are more prevalent in the general population than they had previously thought. Hence, we argue that attribution research cannot be definite one way or the other regarding the base rate underutilization issue.

III. LIABILITY JUDGEMENTS AND BASE-RATE REASONING

8. Our argument from the attribution example was that base-rate information does not logically or mathematically dictate how people should make attributions. Our second argument is that there may be conditions in which it would be highly undesirable to use base rates in making certain decisions. In support of this argument, we describe a case in the use of pure base rates in legal decision making. We call this the blue bus case (ENDNOTE #1). [See Wells (1992) for a fuller treatment of various issues surrounding the blue bus case.] Several versions of the blue bus case have been developed and tested (Wells, 1992), but the generic version that sets the stage for our discussion takes the following form:

    Mrs. Prob is suing the Blue Bus Company for having caused the death
    of her dog. At trial, the following evidence was given:

    Mrs. Prob testified that she was walking her dog on county road #37
    when she heard a large vehicle behind her. She turned around and
    saw a bus swerving recklessly down the road. She jumped out of the
    way but the bus swerved and hit her dog, killing him instantly.
    The incident occurred at 11:40 A.M. The bus continued at a high
    speed down the road. Unfortunately, Mrs. Prob is color blind and
    thus does not know the color of the bus.

    A county transportation official took the stand, was sworn as a
    witness, and testified that there are only two bus companies that
    travel in the county; the Blue Bus Company and the Grey Bus
    Company. Each company uses the road to run empty buses back to
    their stations after dropping off their passengers. Therefore, one
    of these two bus companies had to be responsible for the death of
    Mrs. Prob's dog.

    A second county transportation official took the stand, was sworn
    as a witness, and reported that the Blue Bus Company owned 80% of
    all the buses and that 80% of the county road #37 bus traffic was
    from the Blue Bus Company. The Grey Bus Company owned 20% of all
    the buses and accounted for 20% of the traffic on that road. Mrs.
    Prob's attorney argued that the jury must find the Blue Bus Company
    liable for damages because on the balance of evidence, it was a
    Blue Bus Company bus that killed Mrs. Prob's dog.

9. In civil litigation suits of this type, the basic rule of evidence for deciding against a defendant is that the plaintiff's claim "is more likely than not" or "more likely so than not" (Nesson, 1985, p. 1364), or simply carries a probability of truth greater than .50 (McCormick, 1984; Simon, 1969). Hence, it would appear that the evidence of traffic volume is plenarily sufficient to render a verdict against the defendant.

10. In spite of the apparent sufficiency of the base-rate information (volume of traffic), neither students nor experienced trial judges will rule against the Blue Bus Company when given this case (Wells, 1992) (ENDNOTE #2). In actual cases having base-rate evidence of this type, courts have commonly rendered directed verdicts, in effect throwing the plaintiff's case out of court (see Guenther v. Armstrong Rubber Company, 1969; Smith v. Rapid Transit, Inc., 1945). How should judgment and decision scientists treat this apparent neglect of base-rate information? Is it an error? When judicial rulings dismiss base-rate information of this sort, claim that it is not real evidence, and call it mere "naked statistics," are judges falling prey to a base-rate fallacy?

11. We do not believe that this is a base-rate error and perhaps not an error at all. We agree that the volume-of-traffic base-rate information is clearly relevant to judgments of probability (as do subjects, see ENDNOTE #2), but the use of such information to render verdicts may be judged inappropriate on the basis of sound social policy considerations. Among these is the idea that specific subclasses of citizens might be found legally liable based merely on their membership in a category of people. Suppose, for example, that a male driver had a single-car accident on a particular side street at 3:00 A.M. and that base-rate data indicated that over 97% of male drivers who have single-car accidents at 3:00 A.M. on side streets are drunk at the time. Would it be appropriate to consider this base-rate alone sufficient for conviction on a drunk driving charge? After all, research on people's threshold probability for reasonable doubt indicates that it lies somewhere between .70 and .90, depending on the method used to measure reasonable-doubt thresholds (see Dane, 1985). Although the 97% figure seems to exceed a reasonable doubt threshold for deciding that this driver was drunk, we suspect that even the strongest advocates for the use of base-rate information in decision making would agree that it is not acceptable to use base-rate information to convict the driver.

12. There may also be other reasons to consider the use of base-rate information to be inappropriate in the blue bus case. The idea of fairness in repeated circumstances, for instance, may be violated by the use of base rates. Over the course of many such cases involving separate suits, the blue bus company will pay in 100% of such cases whereas they are responsible for only 80% of such accidents. In the absence of individuating evidence, the base rate is the best solution to minimizing total errors (error rate = 20%), but the problem is that these errors burden only one party in the dispute, the Blue Bus Company.

13. Our general point about the blue bus case is that there are defensible reasons not only for discounting base-rate information but perhaps even for rejecting it entirely as inappropriate or undesirable for use in reaching decisions regarding liability.

IV. GENERAL CONCLUSION

14. We have given two examples, one concerning attributions and one concerning judgments of liability, in which base-rate information has a questionable status for purposes of making certain types of judgments or decisions. In the case of attribution, manipulations of behavioral base rates do not have to affect attributions according to any logical or mathematical requirements. In the case of legal liability judgments, the use of base rates may produce highly undesirable effects that are unanticipated by purely mathematical considerations. Base-rate information may have a fairly clear role and be somewhat free of questionable assumptions when the subjects' task is one of estimating probabilities or likelihoods, but, as we illustrated with these examples, great care must be exercised in arguing that base-rate information ought to be driving judgments and decisions that are not themselves probabilities, likelihoods, or frequencies.

ENDNOTES:

#1. Readers should not confuse the blue bus problem with the so-called "taxi-cab" problem described by Tversky and Kahneman (1980). The taxi-cab problem requires the subject to produce a Bayesian solution combining base-rate information with individuating information. The blue bus problem is a pure base-rate problem with no individuating information.

#2. It should be noted that this overwhelming reluctance to rule in favor of the plaintiff is not attributable to the students or trial judges having an inappropriately high threshold for liability. The same case in other (non-base-rate) forms yields overwhelmingly high rates of verdicts for the plaintiff even when the mathematical and subjective probabilities are held to the same level of certainty (Wells, 1992). Nor have subjects rejected the base-rate information as irrelevant to their subjective probabilities. Subjects' median and modal estimates of the probability that a Blue Bus Company bus has run over Ms. Prob's dog is .80.

REFERENCES

Bar-Hillel, M. (1980). The base-rate fallacy in probability judgments. Acta Psychologica, 44, 211-233.

Bar-Hillel, M. (1983). The base-rate fallacy controversy. In: Decision making under uncertainty (pp. 39-61), ed. R. W. Scholz. Elsevier Science.

Borgida, E. & Brekke, N. (1981). The base-rate fallacy in attribution and prediction. In J. H. Harvey, W. Ickes & R. F. Kidd (Eds.), New directions in attribution research (Vol. 3, pp. 63-95). Hillsdale, NJ: Erlbaum.

Christensen-Szalanski, J.J.J. & Bushyhead, J.B. (1981). Physicians' use of probabilistic information in a real clinical setting. Journal of Experimental Psychology: Human Perception and Performance, 7, 928-935.

Dane, F.C. (1985). In search of reasonable doubt. Law and Human Behavior, 9, 141-158.

Guenther v. Armstrong Rubber Company, 406 F. 2d. 1315-1318 (3rd Cir. 1969).

Kahneman, D., Slovic, P. & Tversky, A. (Eds.) (1982). Judgment under uncertainty: heuristics and biases. New York: Cambridge University Press.

Koehler, J.J. (1993). The Base Rate Fallacy Myth. PSYCOLOQUY 4(49) base-rate.1.koehler.

McCormick, C. (1984). Handbook of the law of evidence (3rd ed.). St. Paul, MN: West.

Nesson, C. (1985) The evidence or the event? On judicial proof and the acceptability of verdicts. Harvard Law Review 98:1357-1392.

Nisbett, R.E. & Borgida, E. (1975). Attribution and the psychology of prediction. Journal of Personality and Social Psychology, 32, 932-943.

Nisbett, R.E., Borgida, E., Crandall, R. & Reed, H. (1976). Popular induction: Information is not necessarily informative. In J.S. Carroll & J.W. Payne (Eds.), Cognition and social behavior. Hillsdale, NJ: Lawrence Erlbaum Assoc., Inc.

Pollard, P. & Evans, J. St. B.T. (1983). The role of 'representativeness' in statistical inference: A critical appraisal. In thinking and reasoning: Psychological approaches (pp. 107-134), ed. J. St. B.T. Evans. Routledge & Kegan Paul.

Probability and inference in the law of evidence [Symposium] (1986). Boston University Law Review 66.

Simon, R. J. (1969). Judges' translation of burdens of proof into statements of probability. Trial Lawyers' Guide, 38, 103-114.

Smith v. Rapid Transit, Inc., 317 Mass. 469, 58 N.E. 2d. 754 (1945).

Tversky, A. & Kahneman, D. (1980). Causal schemas in judgments under uncertainty. Progress in social psychology (pp. 49-72), ed. M. Fishbein. Erlbaum.

Wells, G.L. (1992). Naked statistical evidence of liability: Is subjective probability enough? Journal of Personality and Social Psychology, 62, 739-752.

Wells, G.L. & Harvey, J.H. (1977). Do people use consensus information in making causal attributions? Journal of Personality and Social Psychology, 35, 279-293.

Wells, G.L. & Harvey, J.H. (1978). Naive attributors' attributions and predictions: What is informative and when is an effect an effect? Journal of Personality and Social Psychology, 36, 483-490.


Volume: 5 (next, prev) Issue: 33 (next, prev) Article: 14 (next prev first) Alternate versions: ASCII Summary
Topic:
Article: