Connectionist models purport to model cognitive neuropsychology by means of adaptive linear algebra applied to point neurons. As a theory of cognition, this approach is deficient in several aspects: noncovergence in neurobiological real-time; omission of two topological structures fundamental to the information processing psychology on which connectionist models are based; omission of the local structure of neurobiological processing; omission of actual neuron morphologies, cortical cytoarchitecture, and the cortical orientation response; the inability to perform memory retrieval from point-neuron "weights" in neurobiological real-time; and failure to implement psychological constancy. Cognitive processing by neuronal flows is offered as a viable alternative. Finally, neural nets fail Hempel's test of empirical and systematic import.
2. Connectionist models began with the Perceptron, which came a cropper when Minsky and Papert (1969) pointed out that the system could not perform the XOR operation (the symmetric difference, or disjoint union, of two sets). The symmetric difference $ is one of the two universal operations of algebra (Kostrikin & Shafarevich, 1990: sec. 20), the cartesian product constituting the other one. This may seem to have more significance for mathematics than cognitive theory, but if a full-fledged computational model for cognition is the goal of connectionism, then, lacking the symmetric difference, connectionism falls short. This lacuna further affects the topological structure of connectionism in a critical way. The connectionist model purports to provide a realization of the nodes and edges of information processing psychology. Now the symmetric difference $ enters into the simplicial topology involved in information processing psychology in a significant way (Hoffman, 1980, 1997), viz., as the boundary operator (Henle, 1979). Further it is the basic operation in the ($, not-$) model for dialectical psychology (Hoffman, 1997), which is isomorphic to the information processing psychology on which connectionism is based. Simplicial topology is also relevant to cortical synchronization (Herlihy & Shavit, 1994) and the binding problem for the several modalities. It therefore appears that the two fundamental topological structures in cognition are not to be found in connectionist models.
3. The connectionist model stemmed from Anderson's pioneering work (Anderson, Silverstein, Ritz, & Jones, 1977) on a vector space model for learning, which essentially consists of adaptive linear algebra -- "linear associative nets" -- in Hopfield's (1982) terminology. The neural nets of connectionism are thus at bottom linear algebra, never mind back propagation and other algebraic refinements added later on. Of late, much has been made of dynamical systems theory for psychological phenomena. The differential structure of dynamical systems is relevant, for neuronal inhibition consists essentially of first differences. Without the differential structure and the accompanying neuronal flows, the connectionist is in effect stuck on Zeno's paradox, the cognitive representation taking the form F(x{sub-1}, x{sub-2}, ..., x{sub-n}) rather than F(x{sub-1}, dx{sub-1},..., x{sub-n}, dx{sub-n}). The second form admits neuronal flows in terms of the microscopic structure and action of neurons. (Recall Heraclitus's dictum: Panta rhei, everything flows...) Without a differential structure in their adaptive linear algebraic model, connectionists are forced into point neurons for their "neural nets."
4. Now it is generally agreed among neuroscientists these days that cortical structure is o topographic, as in the retinotopic map from retina to visual cortex o laminar, as in the cytoarchitectural layers of the cortex, and o microcolumnar, as in Hubel and Weisel's cortical hypercolumns. Only the first of these occurs in "neural nets." In connectionism, "neurons" are taken to be point neurons, and "neural" processes rely on calculating numerical correlations, the actual presence of which in the CNS is not only highly unlikely but has never been demonstrated. But cytoarchitecture and distinctive neuron morphologies are generally regarded as essential features of isocortex; indeed it is the breakup of neuronal morphology and the formation of "placque" that is regarded as characteristic of Alzheimer's disease. But there is no way to obtain neuronal arborescences as they are actually constituted from point-neurons. And it is the growth of the neuronal arborescence which keeps pace with memory and learning all through life and which fades in senile dementia and Alzheimer's disease.
5. The rotation of the orientation response from one cortical microcolumn to another is also considered to be an essential feature of the visual system. A model, connectionist or otherwise, that does not take account of this columnar-direction-field nature of neocortex and the associated neuronal morphologies, particularly stellate and pyramidal cells, can hardly offer a valid description of the perceptual component of cognition. Connectionism does not generate in real-time the visual contours of the Figure-Ground Relation.
6. In lieu of processes involving the neuronal arborescences and cytoarchitecture, the connectionist model operates by assigning asymptotic weights to the point neurons. These weights change adaptively each time a new stimulus pattern is presented. The neurobiological correlate of a weight presumably lies in Hebbian synaptic modification. But consider the problem for memory. Each stimulus pattern must have its own set of characteristic weights, which are presumably stacked up in memory for eventual retrieval. The recall process, if the connectionist model is to be a valid cognitive theory, thus requires sorting through all these sets of weights to determine the one appropriate to the current stimulus. Once again we have an insuperable neurobiological problem as well as a combinatorial one. As Hopfield (1982) notes, new memory can be added continuously to (the state correlations) T{sub-ij}. The addition of new memories beyond the capacity (of at most 100 neuron states in his paper) overloads the system and makes all memory states irretrievable unless there is a provision for forgetting old memories. There are clinical pathologies of memory like this, but normal human memory is capable of long term storage. With something like 1010 neurons in the brain, sorting through sets of weights for recall certainly cannot take place in neurobiological real-time. Even if only a single sensory modality is involved, several tens of millions of sets of "weights" would still have to be examined and identified in the real momentary chasing around the sensory system, the limbic system, the midbrain, and frontal and prefrontal cortex.
7. On the other hand, if one takes account of the REAL neural network -- with what neural arborescences, branching axons, layered cytoarchitecture, brain nuclei and gyri, and a multiplicity of neurochemical pathways are trying to tell us -- it appears that true neural networks must consist of flows through neuronal processes mediated and combined in ways consistent with the mathematical structure of equivariant dynamical systems (Hoffman, 1994) and the category of simplicial objects (Hoffman, 1980, 1985, 1997). The presence of this differential structure in the perceptual manifold and the constraints imposed by simplicial topology on the information processing model on which connectionism is based is essential for a valid theory. Such a differential structure, expressing the local versus global nature of brain structure, is lacking in adaptive linear algebra.
8. The psychological constancies are vital if we are not to be moving through a perpetually deforming rubbery surrealistic world. Without the transformation groups to correct for the distortions that the changing environment imposes on percepts, connectionism also faces an insuperable real-time problem. Even for size and shape constancies, such distortions can present themselves as a double infinity of transformations. This is no problem for equivariant dynamical systems (Hoffman, 1994), but would require an infinity of connectionist calculations for constancy invariance on perceived objects. Connectionist applications to special cases that involve the variability embraced by the psychological constancies are limited to a small number of distortions and example calculations.
9. Green mentions Carl Hempel's classical 1952 essay on concept formation in empirical science. This is very close to the condition laid down in Hempel's (1966) book on the philosophy of science that a theory needs not only empirical import (an umbrella, so to speak, over the known facts) but also systematic import. The latter rests upon scientific systematization that conceptualizes diverse connections ("laws" or theoretical principles) among the many different aspects of the empirical world that are relevant. As Green (1998) noted, Hempel (1952: 36) describes this process in terms (Hempel, 1966: 94) reminiscent of the connectionist model: ... the concepts of science are the knots in a network of systematic interrelationships in which laws and theoretical principles form the threads. Yet if one looks in the index of Lau's (1992) collection of key papers on connectionism for "connectionist models", one is referred to "neural networks." The conceptualization in this case is no better than point neurons, and, as we have seen, these fall far short of an adequate description of real neuronal structure and function as it appears in neuropsychology.
Anderson, J. A., Silverstein, J. W., Ritz, S. A., & Jones, R. S. (1977) Distinctive features, categorical perception, and probability learning: Some applications of a neural model. Psychological Review 84: 413-451.
Churchland, P. (1996) Reconstructing consciousness. Eighteenth Annual Conference of the Cognitive Science Society.
Green, CD. (1998) Are Connectionist Models Theories of Cognition? PSYCOLOQUY 9(4) ftp://ftp.princeton.edu/pub/harnad/Psycoloquy/1998.volume.9/ psyc.98.9.04.connectionist-explanation.1.green
Hempel, C. G. (1952) Fundamentals of Concept Formation in Empirical Science. University of Chicago Press.
Hempel, C. G. (1966) Philosophy of Natural Science. Prentice-Hall
Henle, M. (1979) A Combinatorial Introduction to Topology. Freeman
Herlihy, M. & Shavit, N. (Dec 1994) Applications of algebraic topology to concurrent computation. SIAM News, 10-12.
Hoffman, W. C. (1980) Subjective geometry and geometric psychology. Internat. Journal of Mathematical Modelling 1: 349-367.
Hoffman, W. C. (1985) Some reasons why algebraic topology is important in neuropsychology: Perceptual and cognitive systems as fibrations. Internat. J. of Man-Machine Studies 22: 613-650.
Hoffman, W. C. (1994) Equivariant dynamical systems: A formal model for generation of arbitrary shapes. In: Shape in Picture, O. Ying-Lie et al. (Eds.). Springer-Verlag.
Hoffman, W. C. (1997) Mind and the Geometry of Systems. In: Two Sciences of Mind, S. O'Nualla'in, P. McKevitt, & E. Mac Aogain (Eds.). John Benjamins.
Hopfield, J. (1982) Neural networks and physical systems with emergent collective computational abilities. Proc. National Acad. of Sciences 70: 2554-2558.
Kostrikin, A. I. & Shafarevich, I. R. (Eds.) Algebra I. Springer-Verlag.
Lau, D. (1992) Neural Networks: Theoretical Foundations and Analyses. Institution of Electrical & Electronic Engineers.
Minsky, M. & Papert, S. (1969) Perceptrons: An introduction to computational geometry. Cambridge MA: MIT Press (Reissued in an Expanded Edition, 1988).