This book introduces a new neural network model for categorization and learning in neural networks based on ideas from neurobiology, psychology, and engineering. CALM (Categorizing And Learning Module)is both modular and modulatory. It is argued that modularity is important in overcoming many of the problems and limitations of current neural networks.
1.1 This book introduces a new neural network model, CALM, for categorization and learning in neural networks. CALM is based on ideas from neurobiology, psychology, and engineering. It defines a neural network paradigm that is both modular and modulatory. CALM stands for Categorizing And Learning Module and it may be viewed as a building block for neural networks. The internal structure of the CALM module is inspired by the neocortical minicolumn. Several of these modules are connected to form an initial neural network architecture. Throughout the book it is argued that modularity is important in overcoming many of the problems and limitations of current neural networks. Another pivotal concept in the CALM module is self-induced arousal, which may modulate the local learning rate and noise level.
1.2 The concept of arousal has roots in both biology and psychology. In CALM, this concept underlies two different modes of learning: elaboration learning and activation learning. Mandler and coworkers have conjectured that these two distinct modes of learning may cause the dissociation of memory observed in explicit and implicit memory tasks. A series of simulations of such experiments demonstrates that arousal-modulated learning and categorization in modular neural networks can account for experimental results with both normal and amnesic patients. In the latter case, pathological but psychologically accurate behavior is produced by "lesioning" the arousal system of the model. The behavior obtained in this way is similar to that in patients with hippocampal lesions, suggesting that the hippocampus may form part of an arousal system in the brain.
1.3 Another application of CALM to psychological modelling shows how a modular CALM network can learn the word superiority effect for letter recognition. As an illustrative practical application, a small model is described that learns to recognize handwritten digits.
2.1 The book contains a concise introduction to genetic algorithms, a new computing method based on the metaphor of biological evolution that can be used to design network architectures with superior performance. In particular, it is shown how a genetic algorithm results in a better architecture for the digit-recognition model.
2.2 In five appendices, the role of modularity in parallel hardware and software implementations is discussed in some depth. Several hardware implementations are considered, including a formal analysis of their efficiency on transputer networks and an overview of a dedicated 400- processor neurocomputer built by the developers of CALM in cooperation with Delft Technical University. One of the appendices is dedicated to a discussion of the requirements of simulators for modular neural networks.
3.1 The book ends with an evaluation of the psychological and biological plausibility of CALM models and a discussion of generalization, representational capacity of modular neural networks, and catastrophic interference. A series of simulations and a detailed analysis of Ratcliff's simulations of catastrophic interference show that in almost all cases interference can be attributed to overlap of hidden-layer representations across subsequent blocks of stimuli. It is argued that introducing modularity, or some other form of semidistributed representations, may reduce interference to a more psychologically plausible level.
Murre, J.M.J. (1992) Learning and Categorization in Modular Neural Networks. Harvester Wheatsheaf/Erlbaum