| Title & Author | Abstract | |
|---|---|---|
| 5(46) | SUBSYMBOLIC NATURAL LANGUAGE PROCESSING:
AN INTEGRATED MODEL OF SCRIPTS, LEXICON, AND MEMORY [Cambridge, MA: MIT Press, 1993 15 chapters, 403 Pages] Precis of Miikkulainen on Language-Network Risto Miikkulainen Department of Computer Sciences The University of Texas at Austin Austin, TX 78712 risto@cs.utexas.edu |
Abstract:
Distributed neural networks have been very successful in
modeling isolated cognitive phenomena, but complex high-level
behavior has been amenable only to symbolic artificial intelligence
techniques. Aiming to bridge this gap, this book describes
DISCERN, a complete natural language processing system implemented
entirely at the subsymbolic level. In DISCERN, distributed neural
network models of parsing, generating, reasoning, lexical
processing and episodic memory are integrated into a single system
that learns to read, paraphrase, and answer questions about
stereotypical narratives. Using DISCERN as an example, a general
approach to building high-level cognitive models from distributed
neural networks is introduced, and the special properties of such
networks are shown to provide insight into human performance. In
this approach, connectionist networks are not only plausible models
of isolated cognitive phenomena, but also sufficient constituents
for generating complex, high-level behavior.
Keywords: computational modeling, connectionism, distributed neural networks, episodic memory, lexicon, natural language processing, scripts. |
| 5(56) | HIERARCHICAL FEATURE MAPS AND BEYOND
Book review of Miikkulainen on Language-Network Ben Goertzel Computer Science Department Waikato University Hamilton, New Zealand goertzel@waikato.ac.nz |
Abstract:
Hierarchical self-organizing feature maps, as implemented
in the context of DISCERN, represent an important step toward a
computational model of human memory. However, as Miikkulainen
points out, these maps lack the ability to incorporate novel
information on an ongoing basis (1993). It is suggested that in
order to transcend this limitation the distinction between memory
and processing will have to be eliminated in favor of a more
system-theoretic point of view.
Keywords: computational modeling, connectionism, distributed neural networks, episodic memory, lexicon, natural language processing, scripts. |
| 5(57) | BIOLOGICAL CONSTRAINTS AND THE REPRESENTATION OF
STRUCTURE IN VISION AND LANGUAGE Book Review of Miikkulainen on Language-Network Shimon Edelman Dept. of Applied Mathematics & Computer Science The Weizmann Institute of Science Rehovot 76100, ISRAEL edelman@wisdom.weizmann.ac.il |
Abstract:
The computational building blocks of biological
information processing systems are highly interconnected networks
of simple units with graded overlapping receptive fields, arranged
in maps. In view of this basic constraint, it is proposed that the
present stage in the study of cognition should concentrate on
gaining understanding of the cognitive system at the level of the
distributed computational mechanism. The model of script
understanding introduced in the Miikkulainen's (1993) book appears
promising, both because it treats seriously the question of the
architecture of the language processor, and because its
architectural features resemble those used in modeling other
cognitive modalities such as vision.
Keywords: computational modeling, connectionism, distributed neural networks, episodic memory, lexicon, natural language processing, scripts. |
| 5(77) | NARROWING THE GAP: MIIKKULAINEN AND THE
CONNECTIONIST MODELING OF LINGUISTIC COMPETENCE Book Review of Miikkulainen on Language-Network Paul Deane Cognitive Modalities Project Dataware Technologies Ottawa, Canada an995@freenet.carleton.ca |
Abstract:
Despite its early promise, connectionism has had a
minimal impact upon linguistic theory -- largely because it has
proved very difficult to scale neural network models up to the size
necessary to handle realistic linguistic data. Miikkulainen's
(1993) work on script comprehension demonstrates that such
scaling-up is possible. Miikkulainen's technique -- the
construction of a complex system incorporating multiple neural nets
-- is biologically plausible and replicates important
psycholinguistic findings about language processing. However,
Miikkulainen's system lacks the properties which most linguists
would consider diagnostic of human language. Its simple
template-matching system lacks grammatical structure (morphology
and syntax), and supports neither generativity of form nor
compositionality of semantics. These difficulties can probably be
overcome in future work, but will require connectionists to address
the mathematical and theoretical issues inherent in
multiple-network models of the sort Miikkulainen employs.
Keywords: computational modeling, connectionism, distributed neural networks, episodic memory, lexicon, natural language processing, scripts. |
| 5(78) | DISCERN AS A COGNITIVE MODEL AND COGNITIVE
MODELLING FRAMEWORK Book Review of Miikkulainen on Language-Network Ronan Reilly Dept. of Computer Science University College Dublin rreilly@nova.ucd.ie |
Abstract:
In this review, I evaluate the degree to which I feel
Miikkulainen has achieved the goals he set himself in developing
DISCERN. On the positive side, I argue that DISCERN is a
significant achievement in that it demonstrates the feasibility of
building large-scale connectionist systems that exploit the
capabilities of distributed representations. The techniques
embodied in the FGREP component are important and should prove
useful in other applications. On the negative side, however, I
consider that DISCERN represents a missed opportunity. I do not
believe that scripts tell us much that is useful about real natural
language processing, nor do I believe that a system designed to
process scripts can be an informative basis for cognitive
modelling.
Keywords: computational modeling, connectionism, distributed neural networks, episodic memory, lexicon, natural language processing, scripts. |
| 5(79) | THE ROLE OF NEURAL NETWORKS IN COGNITIVE SCIENCE:
EVOLUTION OR REVOLUTION? Book Review of Miikkulainen on Language-Network Itiel E. Dror and Michael J. Young Department of Psychology Benton Hall Miami University Oxford, OH 45056, USA idror@miavx1.acs.muohio.edu youngmj@miavx1.acs.muohio.edu |
Abstract:
Can the neural network approach to understanding cognition
produce a revolution within cognitive science and re-shape our basic
conceptualization of cognition and mind? Miikkulainen's (1993)
efforts in developing a neural network system using results of
symbolic research clearly demonstrate the extra explanatory power of
the neural network approach. However, this line of neural network
research limits their possible role in cognitive science.
Keywords: neural networks, philosophy of language, symbolic models of cognition |
| 5(85) | STORAGE AND REORGANIZATION IN EPISODIC MEMORY Reply to Goertzel on Language-Network Risto Miikkulainen Department of Computer Sciences The University of Texas at Austin Austin, TX 78712 risto@cs.utexas.edu |
Abstract:
Goertzel's (1994b) review points out that the episodic
memory model of DISCERN (Miikkulainen, 1993, 1994) lacks certain
fundamental properties of human memory, such as a capability for
reorganization, and suggests that the distinction between memory
and processing must be eliminated. His points are well taken, but
in light of the emerging understanding of the episodic memory
system in the brain, I will argue that such a distinction is
justified.
Keywords: computational modeling, connectionism, distributed neural networks, episodic memory, lexicon, natural language processing, scripts. |
| 5(86) | REPRESENTATION OF STRUCTURE ON LINGUISTIC MAPS
Reply to Edelman on Language-Network Risto Miikkulainen Department of Computer Sciences The University of Texas at Austin Austin, TX 78712 risto@cs.utexas.edu |
Abstract:
In his review of Subsymbolic Natural Language Processing
(Miikkulainen, 1993, 1994), Edelman (1994) makes several useful
analogies between language processing and vision. His main
argument is that approaches based on common information processing
principles in the brain, such as maps and receptive fields, are
more likely to lead to insights into human cognition. I very much
agree with this idea, and discuss a few concrete ways in which
language processing models can benefit from principles in use in
current visual processing models.
Keywords: computational modeling, connectionism, distributed neural networks, episodic memory, lexicon, natural language processing, scripts. |
| 5(87) | MODULARITY OF MIND, CEREBRAL LOCALISATION AND
CONNECTIONIST NEUROPSYCHOLOGY Book Review of Miikkulainen on Language-Network Robert William Kentridge Psychology Department Univerity of Durham Durham DH1 3LE England robert.kentridge@durham.ac.uk |
Abstract:
Although Miikkulainen's DISCERN (1993) is a model of
cognition and not the brain, its completely connectionist
construction and similarities between the effects of damage on its
behaviour and the neuropsychology of dyslexia may tempt us to treat
it as a brain model. In particular, we may draw parallels between
the architecture of DISCERN and localisation of function in the
cortex. I argue that "modularity" of the brain may be organised in
terms of time-scales as well as space. Forms of "brain-damage" in
connectionist neuropsychology may differ profoundly from those
causing similar effects in people.
Keywords: computational modeling, connectionism, distributed neural networks, episodic memory, lexicon, natural language processing, scripts. |
| 5(88) | REPRESENTATION IN MODULAR NETWORKS
Book review of Miikkulainen on Language-Network Richard Cooper Department of Psychology University College London London WC1E 6BT, United Kingdom r.cooper@psychol.ucl.ac.uk |
Abstract:
The key feature in the success of DISCERN (1993) is not
its connectionist underpinnings but the modularity inherent in its
design. This modularity leads to the implementation of a version of
Fodor's language of thought hypothesis. The question for
connectionists, then, concerns whether such a language of thought
is inevitable in modular connectionist systems.
Keywords: computational modeling, connectionism, distributed neural networks, episodic memory, lexicon, natural language processing, scripts. |
| 6(02) | SUBSYMBOLIC PROCESSING OF EMBEDDED STRUCTURES
Reply to Deane on Language-Network Risto Miikkulainen Department of Computer Sciences The University of Texas at Austin Austin, TX 78712 risto@cs.utexas.edu |
Abstract:
Deane (1994) points out that the template-matching
approach of DISCERN (Miikkulainen, 1993; 1994a) cannot scale up to
realistic linguistic input without fundamental modifications such
as a stack or a short-term memory. I agree with this observation
and will briefly discuss a more sophisticated parsing model that
employs some of the methods Deane suggests.
Keywords: computational modeling, connectionism, distributed neural networks, episodic memory, lexicon, natural language processing, scripts. |
| 6(03) | COMPUTATIONAL CONSTRAINTS AND THE
ROLE OF SCRIPTS IN STORY UNDERSTANDING Reply to Reilly on Language-Network Risto Miikkulainen Department of Computer Sciences The University of Texas at Austin Austin, TX 78712 risto@cs.utexas.edu |
Abstract:
The computational constraints obtained from symbolic
models (as well as large, integrated connectionist models) can
serve to guide research in subsymbolic cognitive science where
neuroscience and psycholinguistics do not provide enough data. One
such idea is scripts, which turns out to have a natural and
powerful implementation in the subsymbolic framework as
regularities in experience. Scripts serve an important role in
story processing in that they allow us to leave out details and
focus on the important events. Story processing beyond scripts,
however, seems to involve mechanisms whose connectionist
implementation would require a high-level monitor on top of the
statistical components.
Keywords: computational modeling, connectionism, distributed neural networks, episodic memory, lexicon, natural language processing, scripts. |
| 6(04) | SYMBOLIC AND SUBSYMBOLIC COGNITIVE SCIENCE
Reply to Dror & Young on Language-Network Risto Miikkulainen Department of Computer Sciences The University of Texas at Austin Austin, TX 78712 risto@cs.utexas.edu |
Abstract:
Symbolic and subsymbolic cognitive science can be seen as
not competing but complementary approaches, serving different
roles. Even though they are perhaps based on incompatible
foundations, symbolic research can serve as a guideline for
developing subsymbolic models, pointing out ways in which a large
cognitive process could be broken apart and made tractable with
current techniques.
Keywords: computational modeling, connectionism, distributed neural networks, episodic memory, lexicon, natural language processing, scripts. |
| 7(34) | WHERE DOES STORY GRAMMAR COME FROM?
Book Review of Miikkulainen on Language-Network Csaba Pleh and Zsuzsa Kaldy Department of General Psychology Eoetvoes Lorand University P.O. Box 4 H-1378 Budapest, Hungary pleh@izabell.elte.hu Kaldy@izabell.elte.hu |
Abstract:
This commentary discusses some problems that we have
encountered with Miikkulainen's language processing model, DISCERN,
described in his book, Subsymbolic Natural Language Processing: An
Integrated Model of Scripts, Lexicon, Memory (1993). First,
DISCERN uses grammatical analysis in an unclarified manner. It
assumes certain notions like Cases without giving them a
connectionist account, and also presupposes a preliminary syntactic
analysis. Neither the categories nor the temporal relations are
connected to research on human parsing. Second, the relations
between sentence parsing and story parsing are left unclear. Third,
the story material used is extremely constrained. These constraints
(one actor, one plot, etc.) are not trivial and seriously limit the
scope of the model for human story processing.
Keywords: computational modeling, connectionism, distributed neural networks, episodic memory, lexicon, natural language processing, scripts. |