SUBSYMBOLIC NATURAL LANGUAGE PROCESSING:
AN INTEGRATED MODEL OF SCRIPTS, LEXICON, AND MEMORY
[Cambridge, MA: MIT Press, 1993 15 chapters, 403 Pages]
Precis of Miikkulainen on Language-Network

Abstract

Distributed neural networks have been very successful in modeling isolated cognitive phenomena, but complex high-level behavior has been amenable only to symbolic artificial intelligence techniques. Aiming to bridge this gap, this book describes DISCERN, a complete natural language processing system implemented entirely at the subsymbolic level. In DISCERN, distributed neural network models of parsing, generating, reasoning, lexical processing and episodic memory are integrated into a single system that learns to read, paraphrase, and answer questions about stereotypical narratives. Using DISCERN as an example, a general approach to building high-level cognitive models from distributed neural networks is introduced, and the special properties of such networks are shown to provide insight into human performance. In this approach, connectionist networks are not only plausible models of isolated cognitive phenomena, but also sufficient constituents for generating complex, high-level behavior.

Keywords

References

• Baddeley, A.D. (1976) The Psychology of Memory. New York: Basic Books.
• Bower, G.H., Black, J.B. and Turner, T.J. (1979) Scripts in memory for text. Cognitive Psychology, 11:177-220.
• Caramazza, A. (1988) Some aspects of language processing revealed through the analysis of acquired aphasia: The lexical system. Annual Review of Neuroscience, 11:395-421.
• Coltheart, M., Patterson, K. and Marshall, J.C., editors (1988) Deep Dyslexia. London; Boston: Routledge and Kegan Paul. Second edition.
• Elman, J.L. (1990) Finding structure in time. Cognitive Science, 14:179-211.
• Graesser, A.C., Gordon, S.E. and Sawyer, J.D. (1979) Recognition memory for typical and atypical actions in scripted activities: Tests for the script pointer+tag hypothesis. Journal of Verbal Learning and Verbal Behavior, 18:319-332.
• Kohonen, T. (1989) Self-Organization and Associative Memory. Berlin; Heidelberg; New York: Springer. Third edition.
• Kolodner, J.L. (1984) Retrieval and Organizational Strategies in Conceptual Memory: A Computer Model. Hillsdale, NJ: Erlbaum.
• Miikkulainen, R. (1993) Subsymbolic Natural Language Processing: an Integrated Model of Scripts, Lexicon, and Memory. Cambridge MA: MIT.
• McCarthy, R.A. and Warrington, E.K. (1990) Cognitive Neuropsychology: A Clinical Introduction. New York: Academic Press.
• Postman, L. (1971) Transfer, interference and forgetting. In Kling, J.W., and Riggs, L.A., editors, Woodworth and Schlosberg's Experimental Psychology, 1019-1132. New York: Holt, Rinehart and Winston. Third edition.
• Rumelhart, D.E. and McClelland, J.L. (1986) On learning past tenses of English verbs. In Rumelhart, D.E., and McClelland, J.L., editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume 2, 216--271. Cambridge, MA: MIT Press.
• Rumelhart, D.E., Hinton, G.E. and Williams, R.J. (1986) Learning internal representations by error propagation. In Rumelhart, D.E. and McClelland, J.L., editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume 1, 318-362. Cambridge, MA: MIT Press.
• Sejnowski, T. J., and Rosenberg, C. R. (1987) Parallel networks that learn to pronounce English text. Complex Systems, 1:145--168.
• Schank, R.C. and Abelson, R.P. (1977) Scripts, Plans, Goals, and Understanding: An Inquiry into Human Knowledge Structures. Hillsdale, NJ: Erlbaum.
• Williams, M.D. and Hollan, J.D. (1981) The process of retrieval from very long-term memory. Cognitive Science, 5:87--119.