Raftopoulos (1998) claims that I (Green 1998) argued that every node in a connectionist network must be semantically interpretable, and that I rely crucially on an untenable distinction between theory and observation to make my argument run. I reply by showing that neither of these claims is correct, and that Raftopoulos's case against my argument does not appear to be coherent.
2. First, Raftopoulos makes two distinct and significant misinterpretations of my position concerning (a) the question of the semantic interpretation of nodes and connections and (b) my argument's alleged reliance on the theory/observation distinction. Second, Raftopoulos himself endorses a variety of different positions, some inconsistent with each other and some redundant or irrelevant to the argument at hand. Time and space prevent me from responding to Raftopoulos' commentary point by point, but I believe that the elaboration of these two difficulties will give a strong indication of where I think his critique goes wrong.
3. Raftopoulos claims that my argument is that "the units of neural networks that use distributed representations do not have semantic content.... [and that] explanations of cognition must be in terms of propositional attitudes, such as beliefs desires expectations, and so forth." (para. 2). This is becoming a standard misinterpretation of my position, one that I have dealt with in replies to other commentaries. To repeat briefly: the argument presented in my target article has nothing whatever to do with the intentionality (or lack thereof) of nodes and connections. That is the gist of Fodor & Pylyshyn's (1988) argument, from which I explicitly distanced myself in the closing paragraphs of my target article. My own argument instead concerns what it is that nodes and connections might CORRESPOND to in the phenomena they are built to model, not with what (if anything) they might REPRESENT. Fodor & Pylyshyn could be dead wrong (as most connectionists believe them to be) and my argument would remain untouched. Indeed, my argument has nothing SPECIFICALLY to do with cognitive science. It applies equally well to all sciences; it is just that connectionist cognitive science has been poorer than most sciences at meeting the challenge I pose.
4. Raftopoulos attempts -- at first hesitantly (para. 8), but later more boldly (para. 13) -- to invoke the legendary difficulties of the theory/observation distinction (TOD) against my argument. This would only be effective if (a) he presented a specific account of the TOD in which he could ground his own claims, and (b) if I had actually claimed that the problem with connectionism is that the nodes and connections MUST correspond to observable entities. First, he does not present such an account, so we have no idea what position he endorses with respect to the TOD (and it therefore floats threateningly above the critique like a sort of phantom hammer that cannot be subjected to scrutiny itself). Second, I have no interest in defending the claim attributed to me that nodes and connections must correspond to something observable. I am quite happy to entertain the possibility of unobservable entities, provided they are well argued for and coherently conceived. Indeed, I regard beliefs and desires to be just such entities, and I do not believe that their inherent unobservability is the major weakness of "folk psychology," whatever other problems it might have.
5. Having misinterpreted my position, Raftopoulos's own position is somewhat hard to pin down. At some points he seems wholly to accept my suggestion (not that I am unwilling to entertain others, mind you) that nodes and connections correspond to neurons. If so, nothing more need be said. We are in agreement. But Raftopoulos seems to think that in adopting this position he is somehow refuting, or at least repudiating, my own. I do not see why. At other times, he seems to argue, as have some previous commentators, that the real theoretical meat of connectionism resides not with nodes and connections but at some "higher" level of analysis, a level that might emerge from cluster analysis or factor analysis or some such.
6. I have discussed this at length in replies to several other commentaries. In brief, my response is: if these "higher" entities are the entities of theoretical interest to connectionists, then THEY are the core of the theory, and the nodes and connections are incidental. But then it is not clear what remains in such theories to justify calling them "connectionist," unless the argument is made that these higher entities can emerge ONLY from connectionist networks. But even here the connectionist is trapped, because the question that then arises is: "and what makes you believe that 'real' (in vivo) cognition is mediated by a real connectionist network?" Here Raftopoulos, like so many other connectionists (most especially some of the very ones he cites: Smolensky 1995; Clark 1995) is caught between wanting to say, on the one hand, that connectionist networks give rise to SOMETHING LIKE beliefs and desires, and recognizing, on the other, that biting the bullet here will result in their having to accept something very much like Fodor & Pylyshyn's implementationalism. So they attempt to back away from this precipice, arguing that there is something special about nodes and connections that symbolic models cannot provide. Fine, but then it is incumbent upon them to be explicit about just what this something special is. My target article was intended to put their feet to the fire on precisely this issue. Connectionism is young, but it has had its period of grace. It is time to fish or cut bait, ontologically.
7. Confusing matters even further is Raftopoulos's adoption of the position that connectionist networks are models in the way that animals are considered models in medical research; viz., that they are not simulations, per se, but rather display new empirical phenomena to be investigated for the light that they shed on the human case by virtue of their presumed similarities to humans (para. 4). This, too, I have discussed in replies to earlier commentaries. I think I have since developed a more sophisticated understanding of the difference between connectionist models and animal models, though my basic position has not changed. We have strong a priori reasons to believe that animals are going to react in ways similar to us under various physical conditions. These reasons are evolutionary and physiological, in the main. They are not unassailable reasons, however. One of the strongest arguments of anti-animal research groups is the litany of errors medical research has made, or would have made, in the past by relying too heavily on this assumption: thalidomide, digitalis, etc.
8. Connectionist models are not like this. First of all, it is arguable that their behavior is not empirical at all. They are formal systems from the outset, and so "observing" their activity to see how it turns out makes such research no more empirical than makes it makes mathematics empirical to "observe" mathematicians at work to see how the derivation they are working on turns out (see Kukla 1989 for the complete version of this argument). Even if one does not buy this, however, there is little to warrant regarding connectionist nets as being homologous to animal models because we simply do not yet have any strong reason to believe that connectionist networks have anything like the relevant similarities to in vivo human cognition that animal physiology have to human physiology. One way out, of course, is to argue from the outset (something to the effect) that nodes and connections correspond to neurons and then to do everything in one's power to abide by the constraints known to govern neurons in one's development of connectionist models of cognition.
9. It is very easy to confuse two quite distinct projects here, especially if one is (perhaps implicitly) convinced from the outset that the activity of connectionist networks is inherently cognitive. One project is simply to investigate what connectionist networks are capable of doing, and then to study how they are able to do these things, using cluster analysis, factor analysis, or whatever other analytic technique one favors. This is NOT cognitive science, however. Cognitive science is the investigation of how cognition works. One of the hypotheses currently taking its turn to prove itself is that cognition is produced by the activity of connectionist networks. One of the strongest pieces of evidence in favor of this hypothesis is the observation that connectionist networks seem to be able to do a lot of the things that 'real' (in vivo) cognitive systems can do. Note that this is not very strong evidence in and of itself. Given that (as the old adage goes) there are infinitely many theories to satisfy any finite set of data, it could well be that connectionist networks are such powerful computers that they can fit most any set of data (though, as an aside, English grammar seems to pose more problems for it than people had once expected).
10. In order to strengthen their claim, it would be nice if advocates of this hypothesis explained to us in exactly what ways they think the parts of the connectionist network are relevantly similar to the "parts" (and I use this terms advisedly) of cognition. One way to go is to say that each node represents a concept or proposition. This is, effectively, the claim of localist connectionists. Advocates of distributed connectionism reject this, and with good reason: the localist connectionist network is just another symbolic model of cognition, and as such does not boast all the advantages that make connectionist computing so much more powerful than its symbolic counterpart (e.g., learning new domains, generalization to new inputs, graceful degradation, etc.).
11. So how do distributed connectionists meet the challenge? One way is to claim that nodes and connections correspond to neurons in the brain, or to related bits of neural anatomy. If they want to reject this as well, so be it; but in its place they must offer us SOME plausible account to convince us that connectionist nets have something more to do with cognition than does, say, the mythical (near-)infinitely large lookup table. We do not reject THAT hypothesis because it does not account for the data. Indeed, it accounts for the data better than any theory we have yet come up with. Instead, we reject it because we think it implausible that such a thing is somehow instantiated in our heads. My challenge to distributed connectionists, then, is similar: tell us precisely in what sense you think connectionist networks are instantiated in our heads. If it is in the organization of our neurons, then abide by the constraints that that assertion imposes upon you. If you think it is in some other way, then lay it out explicitly and let us examine it critically as we would any other scientific theory.
12. Raftopoulos's attempt to invoke the very interesting philosophical work of Hacking (1983) against my argument founders precisely in his conflation of the two distinct projects described in the preceding paragraph. Hacking's classic example is electrons: physicists undertake procedures described as "spraying" electrons, and get the results one would expect if one were actually spraying such entities. Hacking's conclusion: if you can SPRAY them, they must be real, whether or not you can directly observe them. That is, the "reality" of an entity is more dependent on our ability to physically manipulate it than on our ability to see it.
13. I am in essential agreement, but it has no bearing on my argument. It is certainly true that one can manipulate nodes in a connectionist network. Qua ARTIFICIAL connectionist networks, they are undeniably real. That is not the point. The point is that no one has EVER manipulated nodes in a 'real' (in vivo) cognitive system. No one even knows what they are. THAT is the question that must be answered if connectionist networks are to be regarded as scientific theories (or models, if one prefers) of cognitive processes. If they correspond to neurons, then we have an answer, and we can proceed from there. If not, then there is nothing to strongly connect the putative model to the real phenomenon putatively being modeled.
14. I suspect that Raftopoulos would reply that he HAS laid out an account of what the parts of connectionist networks correspond to in "real" cognitive systems, and that the account he has given is quite similar to those offered by Clark (1995) and Smolensky (1995), in their famous replies to Ramsey, Stich & Garon's (1991) argument that connectionism ENTAILS eliminativism with respect to propositional attitudes. (Incidentally, Raftopoulos fails to mention that Stich soon thereafter abandoned the argument in Stich & Warfield, 1995). There is something of an irony in this, because these were PRECISELY the articles that led me to believe that connectionists are really very unsure about what their ontological base is. Although both Smolensky and Clark deny it, it is my belief that, if taken seriously, both their positions lead to something very much like Fodor & Pylyshyn's (1988) implementationalism, despite their best efforts to avoid that drastic (for conectionists) outcome. The only way out of this dilemma is for distributed connectionists to state clearly and categorically what they believe their ontological base to be.
Clark, A. (1995). Connectionist minds. In: Debates on Psychological Explanation, eds. MacDonald, C., and MacDonald, G. , Oxford: Blackwell, 339-356.
Crick, F.H.C., and Asanuma, C. (1986). Certain aspects of the anatomy and physiology of the cerebral cortex. In: Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Vol. 2: Psychological and Biological Models, eds., McClelland, J. L., Rumelhart, D. E., and the PDP Research Group. Cambridge, MA: The MIT Press, 333-371.
Fodor, J. A., and Pylyshyn, Z. W. (1988). Connectionism and cognitive architecture: A critical analysis. Cognition, 28: 3-71.
Green, C. D. (1998). Are connectionist models theories of cognition? PSYCOLOQUY 9(4) ftp://ftp.princeton.edu/pub/harnad/Psycoloquy/1998.volume.9/ psyc.98.9.04.connectionist-explanation.1.green
Hacking, I. (1983). Representing and Intervening. Cambridge: Cambridge University Press.
Kukla, A. (1989). Is AI an empirical science? Analysis, 49, 56-60.
Raftopoulos, A. (1998). Can connectionist theories illuminate cognition: Comment on Green on connectionist-explanation. PSYCOLOQUY 9(24) ftp://ftp.princeton.edu/pub/harnad/Psycoloquy/1998.volume.9/ psyc.98.9.24.connectionist-explanation.21.raftopoulos
Ramsey, W., Stich, S.P., and Garon, J. (1991). Connectionism, eliminativism and the future of folk psychology. In: Philosophy and Connectionist Theory, eds., Ramsey, W., Stich, S. P., and Rumelhart, D. E., Hillsdale, NJ: Erlbaum , 199-228.
Smolensky. P (1995). On the projectible predicates of connectionist psychology: A case for belief. In: Connectionism: Debates on Psychological Explanation, eds. MacDonald, C., and MacDonald, G. , Oxford: Blackwell, 357-394.
Stich, S.P. and Warfield, T. (1995). Reply to Clark and Smolensky: Do connectionist minds have beliefs. In: Connectionism: Debates on Psychological Explanation, eds. MacDonald, C., and MacDonald, G. , Oxford: Blackwell, 395-411.