Second Edition. Oxford: Blackwell (Fall 2002)
William Bechtel
and
Adele Abrahamsen
Two chapters are available on line from Blackwell:
Chapter 8: Connectionism and the Dynamical Approach to Cognition
Chapter 9: Networks, Robots, and Artificial Life
Contents
Preface
1 Networks versus Symbol Systems: Two Approaches to Modeling Cognition
1 A Revolution in the Making?
2 Forerunners of Connectionism: Pandemonium and Perceptrons
3 The Allure of Symbol Manipulation
1.3.1 From logic to artificial intelligence
1.3.2 From linguistics to information
processing
1.3.3 Using artificial intelligence
to simulate human information processing
4 The Disappearance and Reemergence of Network Models
1.4.1 Problems with perceptrons
1.4.2 Re-emergence: The new connectionism
5 New Alliances and Unfinished Business
Sources and Suggested Readings
2 Connectionist Architectures
2.1. The Flavor of Connectionist Processing: A Simulation
of Memory Retrieval
2.1.1 Components of the model
2.1.2 Dynamics of the model
2.1.2.1 Memory
retrieval in the Jets and Sharks network
2.1.2.2 The
equations
2.1.3 Illustrations of the dynamics
of the model
2.1.3.1 Retrieving
properties from a name
2.1.3.2 Retrieving
a name from other properties
2.1.3.3 Categorization
and prototype formation
2.1.3.4 Utilizing
regularities
2.2 The Design Features of a Connectionist Architecture
2.2.1 Patterns of connectivity
2.2.1.1 Feedforward
networks
2.2.1.2 Interactive
networks
2.2.2 Activation rules for units
2.2.2.1 Feedforward
networks
2.2.2.2 Interactive
networks: Hopfield networks and Boltzmann machines
2.2.2.3 Spreading
activation vs. interactive connectionist models
2.2.3 Learning principles
2.2.4 Semantic interpretation of
connectionist systems
2.2.4.1 Localist
networks
2.2.4.2 Distributed
networks
2.3 The Allure of the Connectionist Approach
2.3.1 Neural plausibility
2.3.2 Satisfaction of soft constraints
2.3.3 Graceful degradation
2.3.4 Contentaddressable memory
2.3.5 Capacity to learn from
experience and generalize
2.4 Challenges Facing Connectionist Networks
2.5 Summary
Sources and Recommended Readings
3 Learning
3.1 Traditional and Contemporary Approaches to Learning
3.1.1 Empiricism
3.1.2 Rationalism
3.1.3 Contemporary cognitive science
3.2 Connectionist Models of Learning
3.2.1 Learning procedures for twolayer,
feedforward networks
3.2.1.1 Training
and testing a network
3.2.1.2 The
Hebbian rule
3.2.1.3 The
delta rule
3.2.1.4 Comparing
the Hebbian and delta rules
3.2.1.5 Limitations
of the delta rule: The XOR problem
3.2.2 The backpropagation learning
procedure for multilayered networks
3.2.2.1
Introducing hidden units and backpropagation learning
3.2.2.2
Using backpropagation to solve the XOR problem
3.2.2.3
Using backpropagation to train a network to pronounce words
3.2.2.4
Some drawbacks of using backpropagation
3.2.3 Boltzmann learning procedures
for nonlayered networks
3.2.4 Competitive learning
3.2.5 Reinforcement learning
3.3 Some Issues Regarding Learning
3.3.1 Are connectionist systems
associationist?
3.3.2 Possible roles for innate
knowledge
3.3.2.1
Networks and the rationalist-empiricist continuum
3.3.2.2
Rethinking innateness: Connectionism and emergence
Sources and Suggested Readings
4 Pattern Mapping and Cognition
4.1 Networks as Pattern Recognition Devices
4.1.1 Pattern recognition in twolayer
networks
4.1.2 Pattern recognition in multilayered
networks
4.1.2.1 McClelland
and Rumelhart's interactive activation model of word recognition
4.1.2.2 Evaluating
the interactive activation model of word recognition
4.1.3 Generalization and similarity
4.2 Extending Pattern Recognition to Higher Cognition
4.2.1 Smolensky's proposal: Reasoning
in harmony networks
4.2.2 Margolis's proposal: Cognition
as sequential pattern recognition
4.3 Logical Inference as Pattern Recognition
4.3.1 What is it to learn logic?
4.3.2 A network for evaluating validity
of arguments
4.3.3 Analyzing how a network evaluates
arguments
4.3.4 A network for constructing
derivations
4.4 Beyond Pattern Recognition
Sources and Suggested Readings
5Are Rules Required to Process Representation?
5.1 Is Language Use Governed by Rules?
5.2 Rumelhart and McCelland's Model of Past-Tense Acquisition
5.2.1 A pattern associator with Wickelfeature
encodings
5.2.2 Activation function and learning
procedure
5.2.3 Overregularization in a simpler
network: The rule of 78
5.2.4 Modeling U-shaped learning
5.2.5 Modeling differences between
different verb classes
5.3 Pinker and Prince's Argument for Rules
5.3.1 Overview of the critique of
Rumelhart and McClelland's model
5.3.2 Putative linguistic inadequacies
5.3.3 Putative behavioral inadequacies
5.3.4 Do the inadequacies reflect
inherent limitations of PDP networks?
5.4 Accounting for the U-Shaped Learning Function
5.4.1 The role of input for children
5.4.2 The role of input for networks:
The rule of 78 revisited
5.4.3 Plunkett and Marchman's simulations
of past-tense acquisition
5.5 Conclusion
Sources and Suggested Readings
6 Are Syntactically Structured Representations Needed?
6.1 Fodor and Pylyshyn's Critique: The Need for Symbolic
Representations with Constituent
Structure
6.1.1 The need for compositional
syntax and semantics
6.1.2 Connectionist representations
lack compositionality
6.1.3 Connectionism as providing
mere implementation
6.2 First Connectionist Response: Explicitly Implementing Rules and
Representations
6.2.1 Implementing a production system in a network
6.2.2 The variable
binding problem
6.2.3 Shastri and Ajjanagadde's connectionist
model of variable binding
6.3 Second Connectionist Response: Implementing Functionally
Compositional Representations
6.3.1 Functional vs. concatenative compositionality
6.3.2 Developing compressed representation
using Pollack's RAAM networks
6.3.3 Functional
compositionality of compressed representations
6.3.4 Performing operations
on compressed representations
6.4 Third Connectionist Response: Employing Procedural Knowledge with
External Symbols
6.4.1 Temporal dependencies in processing language
6.4.2
Achieving short term memory with simple recurrent networks
6.4.3 Elman's first study: Learning
grammatical categories
6.4.4 Elman's second study: Respecting
dependency relations
6.4.5 Christiansen's extension: Pushing
the limits of SRNs
6.5 Using External Symbols to Provide Exact Symbol Processing
6.6
Clarifying the Standard: Systematicity and Degree of Generalizability
6.7 Conclusion
Sources and Suggested Readings
7 Simulating Higher
Cognition: A Modular Architecture for Processing Scripts
7.1 Overview of
Scripts
7.2 Overview of Miikkulainen's DISCERN System
7.3 Modular
Connectionist Architectures
7.4 FGREP: An
Architecture That Allows the System to Devise Its Own Representations
7.4.1 Why FGREP?
7.4.2 Exploring FGREP in a simple sentence parser
7.4.3
Exploring representations for words in categories
7.4.4 Moving to multiple
modules: The DISCERN system
7.5 A Self-organizing Lexicon using Kohonen
Feature Maps
7.5.1 Innovations in lexical design
7.5.2 Using Kohonen feature maps in
DISCERN's lexicon
7.5.2.1 Orthography: From high-dimensional vector
representations to map units
7.5.2.2 Associative
connections: From the orthographic map to the semantic map
7.5.2.3
Semantics: From map unit to high-dimensional vector
representations
7.5.2.4 Reversing direction: From semantic
to orthographic representations
7.5.3 Advantages of Kohonen feature maps
7.6 Encoding and Decoding
Stories as Scripts
7.6.1 Using recurrent FGREP modules in DISCERN
7.6.2
Using the sentence parser and story parser to encode stories
7.6.3 Using
the story generator and sentence generator to paraphrase stories
7.6.4
Using the cue former and answer producer to answer questions
7.7 A Connectionist Episodic Memory
7.7.1 Making Kohonen feature maps hierarchical
7.7.2 How role-binding
maps become self-organized
7.7.3 How role-binding maps become trace
feature maps
7.8 Performance: Paraphrasing Stories and Answering Questions
7.8.1 Training and testing DISCERN
7.8.2 Watching DISCERN paraphrase a
story
7.8.3 Watching DISCERN answer questions
7.9 Evaluating DISCERN
7.10
Paths Beyond the First Decade of Connectionism
Sources and Suggested
Readings
8 Connectionism and Dynamical Approach to Cognition
8.1 Are We on the Road to a Dynamical Revolution?
8.2 Basic
Concepts of DST: The Geometry of Change
8.2.1 Trajectories in state space: Predators and prey
8.2.2
Bifurcation diagrams and chaos
8.2.3 Embodied networks as coupled
dynamical systems
8.3 Using Dynamical Systems Tools to Analyze Networks
8.3.1
Discovering limit cycles in network controllers for robotic insects
8.3.2
Discovering multiple attractors in network models of reading
8.3.2.1
Modeling the semantic pathway
8.3.2.2 Modeling the phonological
pathway
8.3.3 Discovering trajectories in SRNs for sentence processing
8.3.4
Dynamical analyses of learning in networks
8.4 Putting Chaos to Work in Networks
8.4.1 Skarda and Freeman's model of the olfactory bulb
8.4.2
Shifting interpretations of ambiguous displays
8.5 Is Dynamicism a
Competitor to Connectionism?
8.5.1 Van Gelder and Port's critique of
classic connectionism
8.5.2 Two styles of modeling
8.5.3
Mechanistic versus covering law explanations
8.5.4 Representations:
Who needs them?
8.6 Is Dynamicism Complementary to Connectionism?
8.7 Conclusion
Sources and Suggested Readings
9 Networks, Robots, and Artificial Life
9.1
Robots and the Genetic Algorithm
9.1.1 The robot as an artificial lifeform
9.1.2 The genetic algorithm for simulated evolution
9.2 Cellular
Automata and the Synthetic Strategy
9.2.1 Langton's vision: The synthetic strategy
9.2.2 Emergent
structures from simple beings: Cellular automata
9.2.3 Wolfram's four
classes of cellular automata
9.2.4 Langton and 8
at the edge of chaos
9.3 Evolution and Learning in Food-seekers
9.3.1
Overview and study 1: Evolution without learning
9.3.2 The Baldwin effect
and study 2: Evolution with learning
9.4 Evolution and Development in
Khepera
9.4.1 Introducing Khepera
9.4.2 The development of phenotypes from
genotypes
9.4.3 The evolution of genotypes
9.4.4 Embodied
networks: Controlling real robots
9.5 The Computational Neuroethology
of Robots
9.6 When Philosophers Encounter Robots
9.6.1 No
Cartesian split in embodied agents?
9.6.2 No representations in
subsumption architectures?
9.6.3 No intentionality in robots and
Chinese rooms?
9.6.4 No armchair when Dennett does philosophy?
9.7
Conclusion
Sources and Suggested Readings
10 Connectionism and the Brain
10.1
Connectionism Meets Cognitive Neuroscience
10.2 Four Connectionist Models of Brain Processes
10.2.1 What/where streams in visual processing
10.2.2 The role of the
hippocampus in memory
10.2.2.1 The basic design and functions of the
hippocampal system
10.2.2.2 Spatial Navigation in Rats
10.2.2.3 Spatial versus declarative memory accounts
10.2.2.4
Declarative memory in humans and monkeys
10.2.3 Simulating dyslexia in
network models of reading
10.2.3.1 Double dissociations in dyslexia
10.2.3.2 Modeling deep dyslexia
10.2.3.3 Modeling surface dyslexia
10.2.3.4
Two pathways versus dual routes
10.2.4 The computational power of modular structure in
neocortex.
10.3 The Neural Implausibility of Many Connectionist Models
10.3.1 Biologically implausible aspects of connectionist networks
10.3.2 How important is
neurophysiological plausibility?
10.4 Wither Connectionism?
Sources and Suggested Readings
Appendix
A Notation
Appendix B Glossary
Bibliography
Name
Index
Subject Index