Goertzel's (1994b) review points out that the episodic memory model of DISCERN (Miikkulainen, 1993, 1994) lacks certain fundamental properties of human memory, such as a capability for reorganization, and suggests that the distinction between memory and processing must be eliminated. His points are well taken, but in light of the emerging understanding of the episodic memory system in the brain, I will argue that such a distinction is justified.
1. Among the many facets and types of human memory, DISCERN's hierarchical feature maps (Miikkulainen, 1993, 1994) were designed to model the long-term storage of everyday experiences. The main hypothesis was that long-term memories are laid out on maps, similar memories next to each other, and the maps are organized under a hierarchy of abstractions, or schemas. Certain types of memory phenomena emerge automatically from this organization, lending computational support to the hypothesis.
2. DISCERN's memory, however, is not a complete model of human episodic memory. It describes the organization of the memory, how it is possible to arrive at such organization based on examples, and how the memory organization and storage and retrieval processes that operate on it result in behavior similar to that of humans. But it does not describe how the organization can be formed gradually while the memory is being used, and it has no mechanisms for storing novel experiences.
3. An alternative is proposed by Goertzel (1993, 1994a, 1994b): a "dual network" model, where reorganization is done by provisionally swapping nodes (representing concepts in the hierarchy), which can be swapped back if the reorganization turns out to be unsuccessful. This way there is no clear distinction between training and reorganization, or even processing and memory, which leads Goertzel to propose that eliminating such distinctions is necessary to achieve psychologically valid reorganization and learning properties.
4. I would like to take a different view. It should be possible to model high-level episodic memory as a separate storage subsystem and still achieve psychologically valid learning properties. This view is motivated by the emerging understanding of the roles of the hippocampus and cortex in the mammalian memory system.
5. Episodic experiences appear to be temporarily stored in the hippocampus as they come in and, over a period of several weeks to several years, transferred to the cortex for permanent storage (Squire, 1992). Why does the permanent encoding take such a long time? One possible answer is that to achieve large capacity, the cortical memory system must be organized as much as possible according to the similarities in the experience. New traces must be carefully encoded, by gradually modifying the existing structure, in order to avoid disrupting the existing traces (McClelland et al., 1994).
6. The hierarchical feature map model acts very much like the cortical memory subsystem. If the new incoming experiences fit the existing structure well, little reorganization is needed. However, novel experiences would require modifying the map structure, which would need to be performed over time by an external training system, such as the hippocampus.
7. While it is possible to model the cortical memory representations in various ways, for example, with distributed representations (McClelland et al., 1994) and attractors (Hopfield, 1982; Hinton and Shallice, 1991), the essential role of the cortex as long-term storage, separate from perception and (re)organization processes, remains the same. How these processes relate to the hippocampal component is a different question. The hippocampus would have to store perceptual experiences immediately as they come in, without the benefit of the entire existing memory structure. It would have to interact with the cortical component, gradually reorganizing it to take into account each new experience; and once the transfer was complete, it would have to reuse the resources in representing later experiences.
8. The hippocampal processes pose a difficult challenge for connectionist modeling. So far it has not been possible to build a high-level model of human episodic memory with an explicit distinction between cortical and hippocampal components. The learning and reorganization concepts developed by Goertzel in his dual network model may well turn out to be appropriate in this task (see also Moll et al., 1994; O'Reilly and McClelland, 1994; Read et al., 1994, for alternative approaches).
Goertzel, B. (1993). The Evolving Mind. New York: Gordon and Breach.
Goertzel, B. (1994a). Chaotic Logic: Language, Thought and Reality from the Perspective of Complex Systems Science. New York: Plenum.
Goertzel, B. (1994b). Hierarchical Feature Maps and Beyond. PSYCOLOQUY 5(56) language-network.2.goertzel.
Hinton, G.E. and Shallice, T. (1991). Lesioning an Attractor Network: Investigations of Acquired Dyslexia. Psychological Review 98:74-95.
Hopfield, J.J. (1982). Neural Networks and Physical Systems with Emergent Collective Computational Abilities. Proceedings of the National Academy of Sciences, USA, 79:2554-2558.
McClelland, J.L., McNaughton, B.L. and O'Reilly, R.C. (1994). Why there are Complementary Learning Systems in the Hippocampus and Neocortex: Insights from the Successes and Failures of Connectionist Models of Learning and Memory. Technical Report PDP.CNS.94.1, Department of Psychology, Carnegie Mellon University.
Miikkulainen, R. (1993). Subsymbolic Natural Language Processing: An Integrated Model of Scripts, Lexicon, and Memory. Cambridge MA: MIT.
Miikkulainen, R. (1994). Precis of: Subsymbolic Natural Language Processing: An Integrated Model of Scripts, Lexicon and Memory. PSYCOLOQUY 5(46) language-network.1.miikkulainen.
Moll, M., Miikkulainen, R., and Abbey, J. (1994). The Capacity of Convergence-Zone Episodic Memory. In Proceedings of the 12th National Conference on Artificial Intelligence, 68-73. San Mateo, CA: Morgan Kaufmann. (ftp:cs.utexas.edu:pub/neural-nets/papers/moll.convergence- zone.ps.Z).
O'Reilly, R.C. and McClelland, J.L. (1994). Hippocampal Conjunctive Encoding, Storage, and Recall: Avoiding a Tradeoff. Technical Report PDP.CNS.94.4, Department of Psychology, Carnegie Mellon University.
Read, W., Nenov V. I. and Halgren, E. (1994). Role of Inhibition in Memory Retrieval by Hippocampal Area CA3, Neuroscience and Biobehavioral Reviews, 18:55-68.
Squire, L.R. (1992). Memory and the Hippocampus: A Synthesis from Findings with Rats, Monkeys, and Humans. Psychological Review 99:195-231.