Evidence obtained with data collection procedures that satisfy inductive and deductive logic provides a better basis for accepting theories than do introspective data. Objectivity is possible even though observations are theory laden. Metatheoretical discussion is less misleading if the data-[inference]-theory sequence is replaced with a phenomenon-theory-data sequence.
1. Psychologists inevitably make some implicit metatheoretical assumptions in their theoretical discussions and empirical research. One of the positive features of Rakover's (1990) METAPSYCHOLOGY is that in it he makes explicit many of these metatheoretical assumptions. The book is a good account of how the majority of psychologists talk about (a) their theories and data (e.g., the view that theories are inferred from research data) and about (b) their methodological concerns (e.g., we cannot justify accepting a universal statement because induction by enumeration does not provide a logical basis for empirical generalization). It is an informative book for readers who may not be well-informed about the history or philosophy of some issues. Readers may find the book not as satisfactory in some places, however, because Rakover attempts to provide a comprehensive account within limited space. Consequently, readers may find their interest whetted, but not satisfied on some topics. Nonetheless, readers of METAPSYCHOLOGY cannot help but be challenged to think about some hitherto taken-for-granted issues, for example, the way we talk about the theory-data relationship.
2. Rakover (1990) begins his discussion of the relationship between theory and data in Chapter 4 with a description of (1) the criteria of scientific data and (2) the pros and cons of the inductivist, logical positivist, falsificationist and holistic approaches to the theory-data relationship. After identifying examples of the four approaches in psychology, he suggests a recursive, 4-stage account of how psychologists may accumulate scientific knowledge and can make progress toward a unified theory by virtue of the inter-dependent relationship between data and theory. Some issues suggested by the theory-data relationship envisaged in METAPSYCHOLOGY are (a) the problematic epistemological status of "private behavior" (PB), (b) some difficulties of the functional characterization of PB, (c) the role of inductive and deductive logic in theory corroboration, and (d) whether or not objectivity is possible when all observations are theory laden.
3. Rakover recognizes two types of private behavior (PB). The first type, PB-I, consists of experiential events of interest to phenomenologists (viz., sensations, emotions and feelings). The second type, PB-II, includes mental processes of interest to cognitive psychologists and cognitive scientists (e.g., perception, thinking, etc.). A characteristic of the "methodological introspection" (p. 215) approach suggested in METAPSYCHOLOGY is that PB is treated as a theoretical term with an explanatory function (pp. 191-192, 271). At the same time, PB is also "used for confirming and falsifying a theory" (p. 215; see also Rakover, 1993). The epistemological status of PB is therefore rendered ambiguous and problematic for the following reason: it is necessary to test what is said about PB in its capacity as theoretical entity; however, it is improper to use PB itself as evidence in such an endeavor.
4. That problematic dual roles are assigned to PB may be the result of Rakover's acceptance of the research methodology of some cognitive scientists (e.g., Newell & Simon, 1972). It is first necessary to describe two distinctions so as to know more about methodological introspection. The first concerns whether variables intervening between observable stimuli and responses are logical constructs (i.e., descriptive abbreviations) or hypothetical mechanisms (viz., efficacious putative structures and processes; see MacCorquodale & Meehl, 1948). Because a logical construct is not used to refer to any entity with existential import, it does not have any theoretical properties. It is simply a shorthand expression used to economize the description of a functional relationship between observable stimuli and responses. A logical construct can be discarded without changing the substance of a functional relationship. An example is Hull's (1943) "habit strength." A hypothetical mechanism, on the other hand, is a theoretical entity (e.g., the iconic memory store) with specific theoretical properties (viz., a relatively large storage capacity which retains input in its veridical form for a very brief period of time). These theoretical properties render an everyday phenomenon meaningful (e.g., our subjective impression that we can at a glance perceive more than we can recall subsequently; see Sperling, 1960).
5. The second distinction is the difference in methodological emphasis between cognitive psychologists (e.g., Posner & Snyder, 1975; Sternberg, 1969) and cognitive scientists (e.g., Pylyshyn, 1978, 1984; Newell & Simon, 1972). Cognitive psychologists' way of corroborating their theories, as illustrated in Table 1, may best be characterized as a "conjectures and refutations" approach (Popper, 1968), more commonly known as the "hypothetical-deductive" method (see Section IV below). When confronted with a to-be-explained phenomenon (e.g., Event 1 in Table 1), a psychologist proposes a state of affairs which, if true, can make sense of the phenomenon (e.g., Event 2 in Table 1). That is, Events 1 and 2 jointly constitute the "conjecture," or "hypothetical," half of the approach.
Table 1. Schematic Representation of the Conjecture Component and the
First Refutation Subcomponent of the "Conjectures and Refutations"
Approach.
Event 1: To-be-explained Perceive more than
Phenomenon, P. can be recalled.
Event 2: Speculation: Iconic memory
Theory T renders store with certain
P meaningful. properties. [T]
Event 3: Implication I Partial report is
should be true superior to whole
if T is true. report. [I]
Event 4: Do evidential data Is the mean of
and [I] match? partial report
higher than the mean
of whole report?
6. The "refutation" component is made up of two applications of deductive logic. In the first application, cognitive psychologists deduce a theoretical implication of a hypothetical mechanism envisaged in a theory (see [I], in Event 3 in Table 1). An implication is a description of an observable behavioral consequence if what is said about the theoretical properties of a hypothetical mechanism is true. More importantly, the state of affairs depicted in [I] is different from P.
7. What is called "prescription" here is called "prediction" in METAPSYCHOLOGY (as in most theoretical and metatheoretical discussions). However, [I] is really not a forecast of what subjects will (or may) do. Rather, it simply stipulates what an investigator should do when research data take a certain form. An implication is, hence, a prescription as to what SHOULD happen, if the theory is to be tenable. It serves as a criterion of theory acceptance or rejection. Thus, "prescription" is a less misleading term than "prediction."
8. The to-be-corroborated theory [T] and the particular implication [I] used to corroborate [T] are represented as the antecedent and consequent, respectively, of a conditional proposition (e.g., [P1] in either column of Table 2). In the second application of deductive logic, cognitive psychologists decide whether or not [T] is warranted by data in virtue of a syllogism whose major premise is the conditional proposition, [P1]. The minor premise of the syllogism is supplied by research data (see Event 4 in Table 1). The "affirming the consequent" schema is used if data are like [I] (left column of Table 2). The MODUS TOLLENS rule is used if data are not as said in [I] (right column of Table 2).
Table 2. A Schematic Representation of the Argument Form Implicated
in Theory Corroboration.
Affirming the Modus Tollens
Consequent
If [T], then [I]. [P1] If [T], then [I]. [P1]
[I] is the case. [I] is not the case.
[T] is true(?) [T] is not true.
9. Instead of dealing with the theoretical properties of a hypothetical mechanism, cognitive scientists are concerned with the functional architecture of the mind (e.g., Pylyshyn, 1984) in the following way. Cognizers (as participants in Newell & Simon's, 1972, studies were called) are asked to give a running commentary on what goes on in their mind as they solve a problem. A cognitive scientist first works out an analytic algorithm, that is, a speculation about how a human cognizer may utilize a structure which mimics the architecture and function of a computer to solve the problem. A cognizer's verbal report is analyzed to reveal a protocol algorithm. The analytic algorithm is validated when there is a match between the analytic and protocol algorithms, a state of affairs called "strong equivalence" by Pylyshyn (1984). Three observations may be made. First, an analytic algorithm plays the role of a theory. Second, theory corroboration does not implicate a conditional proposition like [P1]. Third, introspection is used in theory validation.
10. In short, while PB in metapsychology is definitely not a logical construct, neither is it a hypothetical mechanism in the sense envisaged by cognitive psychologists because "PB (is) characterized functionally" (p. 277), instead of being validated by finding evidential support for its theoretical properties. Rakover has not provided a detailed account of what methodological introspection is. However, his functional characterization of PB and his faith in the sufficiency of the material realization of PB (as evidence for PB) suggest an affinity between his methodological introspection and Pylyshyn's (1984) "strong equivalence."
11. If the affinity in question is granted, the usefulness of PB depends on the acceptability of introspective data at the methodological level. However, using introspection data as evidence for the unobservable PB raises the following questions: (1) Are cognizers aware of their knowledge-states? (2) Is a cognizer's verbal report a literal description of the cognizer's knowledge-state? (3) How can one tell that a cognizer's verbal report is not a rational reconstruction of what may have occurred? (4) A cognizer's articulateness determines how rich a verbal report is. How reliable is a methodology which depends on subjects' articulateness? (5) As cognizers have to think aloud, they may be forced to solve the problem in a way which is not normally undertaken. Hence, is it reasonable to assume that an introspective report is a description of what is typical of a thinking process? (6) A protocol algorithm is not a literal transcript of a cognizer's verbal report. Rather, it is a researcher's construction based on a cognizer's report. However, the researcher is also the individual who constructs the analytic algorithm. This calls into question the objectivity of the strong equivalence criterion.
12. Methodological introspection is attractive to Rakover because he has reservations about using induction and the hypothetical-deductive method. To begin with, a universal statement (e.g., "All ravens are black") cannot be logically justified on the basis of a particular statement (viz., "Some ravens are black"), no matter how many black ravens have been observed. However, Rakover has in mind induction by enumeration, which is different from the more sophisticated modes of induction exemplified by Cohen and Nagel's (1934) reading of Mill's (1973) Method of Difference, as may be seen from Table 3.
Table 3. A Schematic Representation of Method of Difference
FACTOR
Phenomenon 1 2 3 4
P A B C D
-P -A B C D
13. The two states of affairs represented in Table 3 are (i) Phenomenon P occurs when Factors 1, 2, 3 and 4 are represented by Levels A, B, C, and D, respectively, and (ii) Phenomenon P is absent (i.e., -P) when Factors 1, 2, 3 and 4 are represented at Levels -A, B, C, and D, respectively. That is, the two states of affairs are identical in all aspects but one (viz., that Factor 1 is represented at different levels). Consequently, Factors 2 through 4 can be excluded as explanations of P (Cohen & Nagel, 1934). At the same time, there is no reason to exclude Level A of Factor 1 as an explanation of P. The Method of Difference is superior to induction by enumeration because the former, but not the latter, permits one to exclude recognized alternative explanations.
14. A syllogism involving a conditional proposition is implicated in theory corroboration (see Table 2). To conduct an experiment is to set up the conditions stipulated in an implication of a theory. The data so collected may suggest that [I] is the case (i.e., affirming the consequent) or that [I] is not the case (viz., modus tollens). Rakover questions the usefulness of these two syllogisms.
15. First, the truth value of the antecedent of [P1] is indeterminate when the consequent of [P1] is affirmed because the state of affairs described in [I] may be brought about by mechanisms or events other than what is depicted in [I] (i.e., there are alternative explanations). In other words, positive data do not provide a logical reason for accepting a theory (Rakover, 1990). This is a forceful criticism if the data collection procedure does not permit the exclusion of recognized alternative explanations (e.g., data collected with a nonexperimental method). However, it becomes less forceful if experimental data are involved because experimental designs are based on more sophisticated inductive methods which make it possible to exclude alternative interpretations.
16. The basic structure of an experimental design is Mill's (1973) Method of Difference (Boring, 1954, 1969). With reference to Table 3, Factor 1 is the independent variable whose two levels are A and -A. Factors 2, 3 and 4 are control variables because each is represented at the same level at both levels of Factor 1. Decisions about Factor 1 and its levels are determined by the to-be-corroborated theory (say, T). An experimenter may also use control procedures such as counterbalancing and random assignment of subjects in addition to using control variables. These controls are chosen with reference to (a) theoretical alternatives to T (whenever applicable), and (b) methodological assumptions about the experimental task and procedure. More specifically, they are chosen deliberately to exclude recognized alternative explanations of data. In other words, there are good reasons to take [T] seriously when the consequent of [P1] is affirmed by experimental data, despite the logical limitation of affirming the consequent of a conditional proposition.
17. Suppose that data are different from what is depicted in [I] of [P1]. Psychologists should discard [T] by virtue of MODUS TOLLENS (i.e., the antecedent of a conditional proposition is false if the consequent is false). However, the premise underlying a theory-corroboration investigation is not [P1], but [P2],
IF [T.A], THEN [I]. ... [P2],
in which [T.A] stands for the conjunction of a to-be-corroborated theory [T] and a set of auxiliary assumptions [A]. This is the case because no theory can be tested in isolation. Various assumptions are inevitably implicated in testing a theory, namely, theoretical knowledge in cognate areas and methodological assumptions about the experimental design and task used. In other words, it is the [T.A] conjunction which is refuted when data are not as described in [I], not necessarily T (Rakover, 1990).
18. Suppose that Experimenter E wishes to retain [T] in the face of data which are contrary to what is said in [I] by questioning an assumption in [T.A]. Furthermore, suppose that the putative culprit is a well-established theory in a cognate area (e.g., "a-in-c") included in the set of auxiliary assumptions, [A]. Like the majority of critics of the Popperian approach, Rakover has ignored the fact that, if E questions "a-in-c," certain obligations are imposed on E if E takes conceptual rigor seriously. First, E has to provide an alternative theory (e.g., "a'-in-c") which can handle existing data in the cognate area explained by the original theory, "a-in-c." Second, it is necessary for E to provide new data in support of the new theory, "a'-in-c," by conducting additional experiments. Third, having replaced "a-in-c" with "a'-in-c," E effectively has a new conjunction, [T.A']. E has to conduct new experiments to corroborate the new conjunction, [T.A']. In other words, the inevitable presence of auxiliary assumptions in theory-corroboration experimentation is a fatal defect of the hypothetical-deductive method only if psychologists are restricted to corroborating a theory with one sole experiment. Modifying an auxiliary assumption to retain a theory may be justified if it is warranted by data specially collected to test [T.A'] with a new series of related experiments.
19. In the course of evaluating the possibility of having an "eternal" (p. 351) guideline for evaluating scientific knowledge, Rakover disputes the view that "there exists no sharp demarcation in science between observation and theory" (p. 352) even though empirical observations are theory laden by asserting that "in science we have ... an agreed upon observational domain and accepted theories of observation" (p. 352). He further claims that the problem of relativism may be avoided if scientific progress is characterized as a learning process in which induction, analogical reasoning, confirmation and falsification are used in different stages of the learning process.
20. It is difficult to assess Rakover's argument as a rejoinder to relativism because it is not clear (i) why eternity is implicated in epistemological assessment and (ii) whether or not (or how) objectivity is implicated in the "eternal" guideline. It is also not self-evident how appealing to an agreement among scientists can be an answer to the assertion that there is no objective justification for knowledge claims when observations are theory laden (Gergen, 1991). Moreover, questions about research evaluation cannot be answered properly if the basic theory-data relationship is described in a misleading way.
21. Despite his reservations about the role of induction (by enumeration) in the pursuit of psychological knowledge, Rakover subscribes to the view that psychologists infer their theories from data (viz., the data-[inference]-theory sequence). This is not a good characterization of the theory-data relationship. As may be seen from Table 1, vis-a-vis Theory T, a distinction has to be made between PRIOR DATA (i.e., the to-be-explained phenomenon, P, which is observed before T is postulated) and EVIDENTIAL DATA (viz., data collected for the expressive and exclusive purpose of testing T). More important, prior data differ from evidential data (compare Events 1 and 3 in Table 1), a state of affairs which prevents psychologists from entertaining ad hoc theories. In other words, the data-[inference]-theory sequence is unsatisfactory because the importance of theory justification is not represented.
22. Event 3 in Table 1 may also be used to show that objectivity is possible despite theory-laden observations. It is said in [I] that partial report is superior to whole report. Being an implication of what is hypothesized about the iconic store in [T], [I] is obviously dependent on [T]. However, this is NOT what relativism is about because [I] describes a state of affairs which belongs to a level of discourse more abstract than the sort of theory-laden observations envisaged in relativism.
23. [I] describes a relationship between two events. There are three units of discourse, namely, "partial report," "whole report" and "superior to." The relativists' argument is that the identities of these discourse units are dependent on a theory. However, the underlying theory which renders a report "whole" or "partial" is NOT (need not, as well as should not, be) the theory of iconic memory in question. That is, the identity of the discourse units implicated in evidential data of Theory T is not (and should not be) dependent on T itself, even though (a) the state of affairs depicted in the experimental prescription is dependent on T and (b) the theory-laden discourse units are used in describing the experimental prescription.
24. In conclusion, inductive logic plays a more sophisticated role than envisaged in METAPSYCHOLOGY. The hypothetical-deductive method is defensible because psychologists are not restricted to conducting just one experiment in their theory-corroboration attempt. A less misleading way to represent the theory-data relationship is to describe it as a phenomenon-theory-data sequence. Objectivity is possible even though empirical observations are theory laden because (i) phenomena (DATA PRIOR to theory) and evidential data are different observations and because (ii) the theory underlying the identity of the discourse units implicated in evidential data is not the to-be-corroborated theory.
Boring, E. (1954) The Nature and History of Experimental Control. American Journal of Psychology, 67, 573-589.
Boring, E. (1969) Perspective: Artifact and Control. In R. Rosenthal & R.L. Rosnow (Eds.), Artifacts in Behavioral Research (pp. 1-11). New York: Academic Press.
Cohen, M.R. & Nagel, E. (1934) An Introduction to Logic and Scientific Methods. London: Routledge & Kegan Paul.
Gergen, K. (1991) Emerging Challenges for Theory and Psychology. Theory & Psychology, 1, 13-35.
Hull, C.L. (1943) Principles of Behavior: An Introduction to Behavior Theory. New York: Appleton-Century.
MacCorquodale, K. & Meehl, P.E. (1948) On a Distinction Between Hypothetical Constructs and Intervening Variables. Psychological Review, 55, 95-107.
Mill, J.S. (1973) A System of Logic: Ratiocinative and Inductive. Toronto: University of Toronto Press.
Newell, A. & Simon, H.A. (1972) Human Problem Solving. Englewood Cliff, New Jersey: Prentice-Hall.
Popper, K.R. (1968). Conjectures and Refutations: The Growth of Scientific Knowledge (originally published in 1962). New York: Harper & Row.
Posner, M.I. & Snyder, R.R. (1975) Attention and Cognitive Control. In R.L. Solso (Ed.), Information Processing and Cognition: The Loyola Symposium (pp. 55-85). Hillsdale, N.J.: Erlbaum.
Pylyshyn, Z.W. (1978) Computation Models and Empirical Constraints, Behavioral and Brain Sciences, 1, 93-99.
Pylyshyn, Z.W. (1984) Computation and Cognition, Cambridge, Mass.: The MIT Press.
Rakover, S.S. (1990) Metapsychology: Missing Links in Behavior, Mind, and Science. New York: Solomon Press.
Rakover, S.S. (1993) Precis of Metapsychology: Missing Links in Behavior, Mind, and Science. PSYCOLOQUY 4(55) metapsychology.1.rakover.
Sperling, G. (1960) The Information Available in Brief Visual Presentations. Psychological Monographs, 74(11, Whole No. 498).
Sternberg, S. (1969) Memory-scanning: Mental Processes Revealed by Reaction-time Experiments. American Scientist, 57, 421-457.