The key feature in the success of DISCERN (1993) is not its connectionist underpinnings but the modularity inherent in its design. This modularity leads to the implementation of a version of Fodor's language of thought hypothesis. The question for connectionists, then, concerns whether such a language of thought is inevitable in modular connectionist systems.
1. Miikkulainen's DISCERN system (Miikkulainen, 1993, 1994) is undoubtedly impressive. Rather than singing its praises here, however, I would like to consider how, given its connectionist foundations and modular design, it contributes to the debate between symbolic and connectionist approaches to cognitive modeling. Miikkulainen would have us believe that DISCERN strengthens the connectionists' hand, demonstrating for once and for all that connectionist systems are able to perform the sorts of "high level" complex tasks of which previously only symbolic systems have been capable. To evaluate this position, it is necessary to consider just what aspects of DISCERN lead to its striking performance. Miikkulainen gives a lengthy and careful analysis of the system's performance, but what is the theory at the heart of the DISCERN system?
2. In discussing future avenues for research stemming from the DISCERN project, Miikkulainen suggests many ways in which the individual components might be improved. He notes, for example, that the sentence processing module is limited in terms of its potential syntactic coverage, and suggests extensions which allow relative clauses (though the extension is itself severely limited). Similarly, Miikkulainen considers how the system might be extended to handle pronominal reference within script-based stories, how the representation of concepts might be improved, how the lexicon might be made more flexible, and how the feature maps which implement episodic memory might be improved, and so on. Most research projects raise as many questions as they answer, so such possible extensions should not be seen to undermine the system. Nevertheless, we are left with the question: If DISCERN is a theory of the understanding of prototypical stories, then just what is that theory? What are the theoretical assumptions which underlie the DISCERN system? Which aspects of the system is Miikkulainen firmly committed to? The various revisions which Miikkulainen considers suggest that much of the system is open to further development.
3. Given this, and the assumption that any further development will not have a detrimental affect on the performance of the system, precisely which aspects of DISCERN are responsible for its performance? Two features of the system seem to be particularly important: first, the modular decomposition of the task into a set of simpler tasks, and second, the closely related use of a central lexicon, with representations evolved through Miikkulainen's FGREP technique. Both of these are clearly important advances for the connectionist modeling of tasks with requirements that span functionally distinct areas. Let us consider each in turn.
4. The task decomposition resulting in DISCERN's modularity is certainly intuitive. It is also classical in the sense that it is not so different from previous symbolic models of script understanding. Indeed, modularity seems (perhaps for reasons of communication between modules, as discussed in the following section) to be closely linked to the symbolic school of modeling. However, the arguments of Marr (1982) and Fodor (1983) for some form of modularity hypothesis are not inextricably linked to symbolic notions. Neuropsychological evidence also points to some degree of modularity (see, for example, Shallice, 1988), and again, this is not tied to symbolic notions. So to ignore the success that symbolic modeling has achieved through modularization, simply for the sake of developing connectionist models that eschew everything remotely symbolic would be to throw out the baby with the bath water. The development of modular networks such as Miikkulainen's seems a most sensible way to make progress in understanding higher-level cognitive processes. Indeed, one way of seeing Miikkulainen's system is as a fleshing out, or an operationalization, of the sort of box/arrow diagram often used by cognitive psychologists. This perspective has important consequences, for such diagrams are often criticised as being vague, ill-specified, or even meaningless. Techniques such as Miikkulainen's may play a vital role in rebutting this claim.
5. Let us return to the many further developments suggested by Miikkulainen. Given his suggestions, what can be said about the innards of the individual modules? One obvious question is: What is the role of the underlying technology of each component of the system? Miikkulainen seems to accept that better connectionist technology may emerge for the various components. This is the point of many of the further developments. Thus, we must ask how important is the detailed functioning of each box? Could we, for example, replace one box by a symbolic equivalent without altering the behavior of the system as a whole? Such a symbolic equivalent would need to interface appropriately with each of its neighboring boxes, but given the strong regularities in the distributed representations employed (for example, the use of fixed positions in the feature vectors to encode particular role bindings, and the tendency for each system to yield categorical output) this should not be too problematic.
6. The point is not that I suspect that the innards of all boxes are mere implementational details. The point is that those of us who build computational models owe it to the rest of the cognitive science community to be clear about which aspects of our model are theoretically motivated and which aspects are implementational details, present only for computational completeness. In Miikkulainen's case, it sometimes seems that the only aspect he is committed to on a theoretical basis is the use of connectionist technology.
7. It is tempting to take the extreme position and ask: Why the obsession with implementing everything in connectionist terms? Surely some aspects related to script processing are inherently symbolic. Miikkulainen mentions (as future work) the possibility of allowing multiple active scripts, so that, for example, a robbery script (having one's wallet stolen) and restaurant script might interact, yielding a script in which paying at the restaurant would be troublesome. Surely our knowledge that we cannot pay for something if we have just had our wallet stolen (assuming that all of our cash/credit cards were in the wallet) is not based on a statistical regularity that we have abstracted from past experience (one at least would hope not). This kind of knowledge seems to be based on explicit symbolic reasoning concerning the necessity for cash or credit cards when paying, and the lack of such things as a result of robbery. To attempt to capture such (explicit symbolic) reasoning processes via interacting scripts based only on statistical regularities -- rather than via knowledge rich causal and structural theories of the world -- seems perverse.
8. Overall then, despite DISCERN's impressive performance, and despite the real advances it offers in our understanding of cross-domain cognitive modeling, the theory underlying DISCERN's modules appears patchy. It would seem that DISCERN's convincing performance is more a result of the modularity inherent in the system than the underlying theory of the particular modules, or even their connectionist implementation.
9. The modularization that is fundamental to DISCERN leads one immediately to the question of how such modules communicate. Miikkulainen's solution, involving a central lexicon and distributed representations with units dedicated to particular case roles, looks suspiciously like a language of thought. The fact that most modules produce categorical output reinforces this view. This is not a damning critique, merely a surprising feature for a connectionist account. There is, however, a sense in which such a language flows inevitably from the sort of modularization which Miikkulainen employs.
10. One can quibble about what constitutes a language. To Miikkulainen, a language is something that results from the need to pass a given amount of information along a channel of limited breadth (and the purpose of several of his modules is translation between two such representational forms). In particular, "the recurrent FGREP networks translate between stationary, wide internal representations of complex knowledge (thought), and sequential representations of this knowledge used for transmission through narrow communication channels (language)" (Miikkulainen, 1993, p.288).
11. As such, language to Miikkulainen is necessarily serial, but this is not a defining characteristic of language, so the avoidance of such sequentiality in the representations which pass between Miikkulainen's modules in no way compromises their candidacy for forming a language of thought. The other extreme is to suggest that any system of representation forms a language. That is, if we have symbols (including feature vectors) which stand in place of "concepts", then we have a language. This, however, is not the issue for debate. To many, and especially those who pursue the language of thought hypothesis (e.g., Fodor, 1987), the key to language is systematicity and compositionality (see, e.g., Fodor & Pylyshyn, 1988).
12. A language is compositional if there exist nonatomic sentences in that language and if the meanings of those sentences may be determined from the meanings of their parts and the mode of combination of those parts. A language is systematic if, for any two properties P and Q and any two individuals x and y such that the language can express P(x), P(y) and Q(x), the language can also express Q(y).
13. Are Miikkulainen's feature vectors compositional and systematic? Do they constitute a language of thought? These questions can be answered in the affirmative in two ways: A strong language of thought would require that the representations at all interfaces have a uniform syntax and semantics. For example, all such representations might consist of 60 features ranging from 0 to 1, with units 1-12 representing an agent, units 13-24 representing an act, and so on. A weak language of thought would simply require that the language at each interface be compositional and systematic, but would allow different languages at each interface. It appears to be this second option which is used by DISCERN. The atoms, at all interfaces, have a standard interpretation (given by the central lexicon), but each interface uses a different language tailored to its requirements. Thus, the interface language between the sentence parser and the story parser consists of features of vectors with 60 components as described above. The interface language for the episodic memory (both input and output), however, consists of feature vectors with 84 components (cf. Miikkulainen, 1993, p.121). The first 12 components represent the script, the next 12 represent the track, and so on. Furthermore, because of Miikkulainen's encoding of types and tags, each component of 12 features can be further analyzed in terms of 2 features which represent the particular token and 10 which represent the type of which the corresponding object is a token. Each interface language is compositional in this sense. The languages are also systematic. There is a clear argument structure and this allows us, for any representations (within a single language) P(x), P(y) and Q(x), to extract the representation for x from the representation for Q(x) and replace it with the representation of y, thus obtaining the representation of Q(y). We thus find in DISCERN a weak deployment of the language of thought hypothesis.
14. I am not suggesting that DISCERN merely implements a standard symbolic language. Although the representations are systematic and compositional, there is a crucial difference between DISCERN and standard symbolic systems in the relationship between the representations and the processes which operate over them. In DISCERN, representations, despite being compositional and systematic, are transformed without being interpreted. Within each module, representations remain implicit. It is only between modules that representations are explicit. To what extent this is a necessary consequence of the modularization used by Miikkulainen is open to debate. In any case, it does allow us to interpret the system in a reductionist manner: mental states (in as much as DISCERN can be said to have beliefs, etc.) reduce to brain states (or patterns of activation in/between the various modules).
15. Miikkulainen (1993) argues that his DISCERN system is not a hybrid symbolic/connectionist model, but rather a modular connectionist model. Given the use by the system of systematic, compositional representations, it is not clear that DISCERN is not both modular connectionist and hybrid symbolic/connectionist. The question for connectionists is whether there can exist modular connectionist models which are not also hybrid in the standard symbolic/connectionist sense.
Fodor, J. (1983) The Modularity of Mind: An Essay on Faculty Psychology. Cambridge, MA: MIT Press.
Fodor, J. (1987) Psychosemantics: The Problem of Meaning in the Philosophy of Mind. Cambridge, MA: MIT Press.
Fodor, J. and Pylyshyn, Z. (1988) Connectionism and Cognitive Architecture. Cognition, 28, 3-71.
Marr, D. (1982) Vision. San Francisco: Freeman.
Miikkulainen, R. (1993) Subsymbolic Natural Language Processing: An Integrated Model of Scripts, Lexicon and Memory. Cambridge, MA: MIT Press.
Miikkulainen, R. (1994) Precis of: Subsymbolic Natural Language Processing: An Integrated Model of Scripts, Lexicon and Memory. PSYCOLOQUY 5(46) language-network.1.miikkulainen.
Shallice, T. (1988) From Neuropsychology to Mental Structure. Cambridge, UK: Cambridge University Press.