Can the neural network approach to understanding cognition produce a revolution within cognitive science and re-shape our basic conceptualization of cognition and mind? Miikkulainen's (1993) efforts in developing a neural network system using results of symbolic research clearly demonstrate the extra explanatory power of the neural network approach. However, this line of neural network research limits their possible role in cognitive science.
2. Two issues need to be addressed: First, what is going to be the relationship between neural network research and symbolic research? Second, how is the neural network approach going to affect (and how can it affect) the development of cognitive science and its related fields?
3. Because the two approaches are incompatible, different researchers have usually adopted one approach or the other. This dichotomy within the field of cognitive science seems healthy in the short term, but is problematic and demonstrates a lack of understanding on our part in the long term. In the short term, the dichotomy reveals a liberal, non-dogmatic field that allows the existence of multiple and conflicting approaches for studying cognition. It enables researches to explore alternative approaches to what may become the mainstream of thought in cognitive science. However, in the long term, one needs either to reconcile the two approaches or to adopt one approach as the correct one. If cognitive science is a true science, then it cannot include two incompatible approaches for its computational foundation.
4. The obvious long term development of the field may be that one approach will demonstrate its supremacy by providing a unified theory of cognition and the other will fade away and eventually disappear. However, there are a few possible alternatives that allow both approaches to survive. Both may evolve to take different roles within the field of cognitive science; for example, the symbolic approach may be the language and terminology that we use to describe the general development of the field, but the actual cognitive research and hence the discoveries and findings in the field will emerge through the neural network approach (endnote 1). Other possibilities of conceptualizing the two approaches as appropriate for different levels of description of cognition are also possible. Both approaches may sustain their power as experimental tools for cognitive research, but in this case each approach will be applied to different domains of cognition, that is, cognition will be divided into domains that are symbol based and domains that are neural network based. It is also possible that in the long term both approaches will be reconciled in a more fundamental way, producing so called "hybrid" models.
5. Although it may be too early to reach a verdict or find a solution to the dichotomy between the symbolic and neural network approaches to studying cognition, it is definitely time to start considering this issue more closely. Such a discussion must include a detailed examination of the philosophical foundations of each approach (see Dror & Dascal, in press, for such a discussion). However, the ultimate challenge of both approaches lies in their ability to explain empirical phenomena.
6. Miikkulainen's book is a superb effort in this direction. One of the main strengths of Miikkulainen's work is that he develops and examines a comprehensive and integrated model of a complex high-level cognitive task. By its nature, this type of project must span different paradigms of research, each of which examines various aspects of cognition. Miikkulainen does an excellent job reviewing and integrating such diverse and complex issues.
7. Miikkulainen proposes that the symbolic and neural network approaches are incompatible. This enables the discussion to focus on their comparison rather than on whether they are reconcilable as hybrid models or not. This is a wonderful framework for discussing the role of neural networks in cognitive science.
8. Although Miikkulainen agrees that the two approaches are incompatible, he asserts that many of the results obtained through symbolic research are still valid and can be used as guidelines for developing neural network models of cognitive processing. This view warrants a closer examination: What advantages does it offer? What, if any, are its limitations and dangers? What are the theoretical assumptions that underlie such an assertion? And where might it lead us?
9. The use of symbolic research as a guide for neural network modeling seems, on the face of it, very reasonable. It utilizes the advantages of the neural network approach; for example, the ability to learn from examples, generalize, deal with noise, and gracefully degrade, are not only important for application purposes, but are also fundamental characteristics exhibited by cognitive processes. Furthermore, neural network models are more biologically feasible than symbolic models. The use of symbolic research contributes to this line of research by providing guidelines for neural network modeling. It contributes a rich, well-researched body of knowledge describing cognition.
10. The use of neural networks provides greater explanatory power than the symbolic approach, and thus can give a better account of high-level cognitive phenomena. For example, Miikkulainen uses this line of research to reveal where performance errors occur, how memory becomes overloaded and what happens to representations when this occurs. These are all important and valid points, however, this line of research assigns a limited role for neural networks.
11. Neural networks -- according to this view -- have not established a new foundation for cognitive science; rather, they stand on the foundation laid down by the symbolic research. This foundation is reflected in the conceptual framework that contains the results of symbolic research. Although not the intention, using the results of the symbolic approach entails that we indirectly adopt the conceptual framework that comes with it (e.g., scripts and lexicon). Thus, using symbolic research as a guide for neural network research assigns them a role WITHIN the symbolic conceptual framework. Nevertheless, even in this limited role, neural networks can contribute and play an important role in the evolution of cognitive science.
12. The question with far reaching implications is whether neural networks can establish a new foundation for cognitive science that will bring about a shift in the basic way we conceptualize cognition altogether. That is, Will neural networks have a revolutionary or evolutionary role in cognitive science? Language is an excellent domain for examining this question and illustrating the different and distinct roles neural networks can have in cognitive science.
13. Using the results of the symbolic research as a guide for neural network research seems to be similar to Chomsky's distinction between competence and performance. The symbolic approach has established and characterized the competence, and neural networks will provide the details of the performance. Needless to say, the research about performance will definitely have greater explanatory power and provide insight into competence, and may even modify our understanding of competence itself.
14. It is definitely worth exploring the possibility that neural networks may have a bigger role in cognitive science. In the domain of language, rather than being an alternative approach to performance modeling, neural networks might provide an alternative view of language altogether. Consider, for example, the current Chomskyian conceptualization of language (Chomsky, 1965, 1980, 1986) versus Wittgenstein's view of language as a "form of life" and "language games" (Wittgenstein, 1953).
15. In the Chomskyian conceptualization, language is a set of rules that every competent speaker internalizes. The creative aspect of language is the infinite application of the rules to generate and understand language. Within the symbolic framework, parsers, grammars, and other mechanisms have been developed as explanatory models of language (e.g., Moyne, 1985), though these models do not in any way approach the performance of humans.
16. In contrast, in the Wittgensteinian conceptualization of language there are no rules, but rather "language is part of an activity, or of a form of life" (Wittgenstein, 1953). Language is not learned by acquisition of rules, but through the plurality of "language games" that capture the regularity of social practices.
17. We are not talking here about differences in implementation, but rather about an entirely different conceptualization of what language is all about. Language may be viewed from either of the two approaches (see Winograd, 1972 and 1983, versus Wingorad & Flores, 1986, as examples of two completely different conceptualizations of language, which are parallel to Chomsky's versus Wittgenstein's conceptualization of language). Wittgenstein's view of language as a set of regularities that are not governed by rules and rule following is consistent with the neural network approach (Dror & Dascal, in press).
18. To fully explore the possible impact of the neural network approach on cognitive science, we must abandon traditional symbolic conceptualizations of cognition and mind. Only then can we truly examine what new conceptualizations emerge through the neural network approach. Are neural networks going to produce a revolution in cognitive science? It seems that using results of the symbolic approach research as a guide for neural network modeling is an obstacle to a true and full exploration of this possibility.
1. This idea was suggested to the first author, in a personal communication, by William K. Estes.
Chomsky, N. (1965). Aspects of the Theory of Syntax. MIT Press, Cambridge, MA.
Chomsky, N. (1980). Rules and Representations. Columbia University Press, New York.
Chomsky, N. (1986). Knowledge of Language: Its Nature, Origin, and Use. Praeger Press, New York.
Dror, I.E. & Dascal, M. (in press). Can Wittgenstein help free the mind from rules? The philosophical foundations of connectionism. In D. Johnson & C. Erneling (Eds.), Reassessing the Cognitive Revolution. Oxford University Press.
Miikkulainen, R. (1993) Subsymbolic Natural Language Processing: An Integrated Model of Scripts, Lexicon and Memory. Cambridge, MA: MIT Press.
Miikkulainen, R. (1994) Precis of: Subsymbolic Natural Language Processing: An Integrated Model of Scripts, Lexicon and Memory. PSYCOLOQUY 5(46) language-network.1.miikkulainen.
Moyne, J. (1985). Understanding Language: Man or Machine. Plenum Press, New York.
Winograd, T. (1972). Understanding Natural Language. Academic Press, Orlando, FL.
Winograd, T. (1983). Language as a Cognitive Process. MIT Press, Cambridge, MA.
Winograd, T. & Flores, F. (1986). Understanding Computers and Cognition: A New Foundation for Design. Ablex Publishing, Norwood, NJ.
Wittgenstein, L. (1953). Philosophical Investigations. Basic Blackwell, Oxford.