Connectionism and the Mind

Parallel Processing, Dynamics, and Evolution in networks

Second Edition. Oxford: Blackwell (Fall 2002)

William Bechtel
Adele Abrahamsen


Two chapters are available on line from Blackwell:

Chapter 8: Connectionism and the Dynamical Approach to Cognition

Chapter 9: Networks, Robots, and Artificial Life



1 Networks versus Symbol Systems: Two Approaches to Modeling Cognition
    1 A Revolution in the Making?
    2 Forerunners of Connectionism: Pandemonium and Perceptrons
    3 The Allure of Symbol Manipulation
        1.3.1 From logic to artificial intelligence
        1.3.2 From linguistics to information processing
        1.3.3 Using artificial intelligence to simulate human information processing
    4 The Disappearance and Reemergence of Network Models
        1.4.1 Problems with perceptrons
        1.4.2 Re-emergence: The new connectionism
    5 New Alliances and Unfinished Business
    Sources and Suggested Readings
2 Connectionist Architectures
    2.1. The Flavor of Connectionist Processing: A Simulation of Memory Retrieval
        2.1.1 Components of the model
        2.1.2 Dynamics of the model
   Memory retrieval in the Jets and Sharks network
   The equations
        2.1.3 Illustrations of the dynamics of the model
   Retrieving properties from a name
   Retrieving a name from other properties
   Categorization and prototype formation
   Utilizing regularities
    2.2 The Design Features of a Connectionist Architecture
        2.2.1 Patterns of connectivity
   Feedforward networks
   Interactive networks
        2.2.2 Activation rules for units
   Feedforward networks
   Interactive networks: Hopfield networks and Boltzmann machines
   Spreading activation vs. interactive connectionist models
        2.2.3 Learning principles
        2.2.4 Semantic interpretation of connectionist systems
   Localist networks
   Distributed networks
   2.3 The Allure of the Connectionist Approach
        2.3.1 Neural plausibility
        2.3.2 Satisfaction of soft constraints
        2.3.3 Graceful degradation
        2.3.4 Contentaddressable memory
        2.3.5 Capacity to learn from experience and generalize
    2.4 Challenges Facing Connectionist Networks
    2.5 Summary
    Sources and Recommended Readings

3 Learning
    3.1 Traditional and Contemporary Approaches to Learning
        3.1.1 Empiricism
        3.1.2 Rationalism
        3.1.3 Contemporary cognitive science
    3.2 Connectionist Models of Learning
        3.2.1 Learning procedures for twolayer, feedforward networks
   Training and testing a network
   The Hebbian rule
   The delta rule
   Comparing the Hebbian and delta rules
   Limitations of the delta rule: The XOR problem
        3.2.2 The backpropagation learning procedure for multilayered networks
   Introducing hidden units and backpropagation learning
   Using backpropagation to solve the XOR problem
   Using backpropagation to train a network to pronounce words
   Some drawbacks of using backpropagation
        3.2.3 Boltzmann learning procedures for nonlayered networks
        3.2.4 Competitive learning
        3.2.5 Reinforcement learning
    3.3 Some Issues Regarding Learning
        3.3.1 Are connectionist systems associationist?
        3.3.2 Possible roles for innate knowledge
   Networks and the rationalist-empiricist continuum
   Rethinking innateness: Connectionism and emergence
Sources and Suggested Readings

4 Pattern Mapping and Cognition
    4.1 Networks as Pattern Recognition Devices
        4.1.1 Pattern recognition in twolayer networks
        4.1.2 Pattern recognition in multilayered networks
   McClelland and Rumelhart's interactive activation model of word recognition
   Evaluating the interactive activation model of word recognition
        4.1.3 Generalization and similarity
    4.2 Extending Pattern Recognition to Higher Cognition
        4.2.1 Smolensky's proposal: Reasoning in harmony networks
        4.2.2 Margolis's proposal: Cognition as sequential pattern recognition
    4.3 Logical Inference as Pattern Recognition
        4.3.1 What is it to learn logic?
        4.3.2 A network for evaluating validity of arguments
        4.3.3 Analyzing how a network evaluates arguments
        4.3.4 A network for constructing derivations
    4.4 Beyond Pattern Recognition
Sources and Suggested Readings

5Are Rules Required to Process Representation?
    5.1 Is Language Use Governed by Rules?
    5.2 Rumelhart and McCelland's Model of Past-Tense Acquisition
        5.2.1 A pattern associator with Wickelfeature encodings
        5.2.2 Activation function and learning procedure
        5.2.3 Overregularization in a simpler network: The rule of 78
        5.2.4 Modeling U-shaped learning
        5.2.5 Modeling differences between different verb classes
    5.3 Pinker and Prince's Argument for Rules
        5.3.1 Overview of the critique of Rumelhart and McClelland's model
        5.3.2 Putative linguistic inadequacies
        5.3.3 Putative behavioral inadequacies
        5.3.4 Do the inadequacies reflect inherent limitations of PDP networks?
    5.4 Accounting for the U-Shaped Learning Function
        5.4.1 The role of input for children
        5.4.2 The role of input for networks: The rule of 78 revisited
        5.4.3 Plunkett and Marchman's simulations of past-tense acquisition
    5.5 Conclusion
Sources and Suggested Readings

6 Are Syntactically Structured Representations Needed?
    6.1 Fodor and Pylyshyn's Critique: The Need for Symbolic Representations with Constituent
        6.1.1 The need for compositional syntax and semantics
        6.1.2 Connectionist representations lack compositionality
        6.1.3 Connectionism as providing mere implementation
    6.2 First Connectionist Response: Explicitly Implementing Rules and Representations
        6.2.1 Implementing a production system in a network
        6.2.2 The variable binding problem
        6.2.3 Shastri and Ajjanagadde's connectionist model of variable binding
    6.3 Second Connectionist Response: Implementing Functionally Compositional Representations
        6.3.1 Functional vs. concatenative compositionality
        6.3.2 Developing compressed representation using Pollack's RAAM networks
        6.3.3 Functional compositionality of compressed representations
        6.3.4 Performing operations on compressed representations
    6.4 Third Connectionist Response: Employing Procedural Knowledge with External Symbols
        6.4.1 Temporal dependencies in processing language
        6.4.2 Achieving short term memory with simple recurrent networks
        6.4.3 Elman's first study: Learning grammatical categories
        6.4.4 Elman's second study: Respecting dependency relations
        6.4.5 Christiansen's extension: Pushing the limits of SRNs
    6.5 Using External Symbols to Provide Exact Symbol Processing
    6.6 Clarifying the Standard: Systematicity and Degree of Generalizability
    6.7 Conclusion
Sources and Suggested Readings

7 Simulating Higher Cognition: A Modular Architecture for Processing Scripts
    7.1 Overview of Scripts
    7.2 Overview of Miikkulainen's DISCERN System
    7.3 Modular Connectionist Architectures
    7.4 FGREP:
An Architecture That Allows the System to Devise Its Own Representations
        7.4.1 Why FGREP?
        7.4.2 Exploring FGREP in a simple sentence parser
        7.4.3 Exploring representations for words in categories
        7.4.4 Moving to multiple modules: The DISCERN system
    7.5 A Self-organizing Lexicon using Kohonen Feature Maps
        7.5.1 Innovations in lexical design
        7.5.2 Using Kohonen feature maps in DISCERN's lexicon
   Orthography: From high-dimensional vector representations to map units
   Associative connections: From the orthographic map to the semantic map
   Semantics: From map unit to high-dimensional vector representations
   Reversing direction: From semantic to orthographic representations
        7.5.3 Advantages of Kohonen feature maps
    7.6 Encoding and Decoding Stories as Scripts
        7.6.1 Using recurrent FGREP modules in DISCERN
        7.6.2 Using the sentence parser and story parser to encode stories
        7.6.3 Using the story generator and sentence generator to paraphrase stories
        7.6.4 Using the cue former and answer producer to answer questions
    7.7 A Connectionist Episodic Memory
        7.7.1 Making Kohonen feature maps hierarchical
        7.7.2 How role-binding maps become self-organized
        7.7.3 How role-binding maps become trace feature maps
    7.8 Performance: Paraphrasing Stories and Answering Questions
        7.8.1 Training and testing DISCERN
        7.8.2 Watching DISCERN paraphrase a story
        7.8.3 Watching DISCERN answer questions
    7.9 Evaluating DISCERN
    7.10 Paths Beyond the First Decade of Connectionism
Sources and Suggested Readings

8 Connectionism and Dynamical Approach to Cognition
    8.1 Are We on the Road to a Dynamical Revolution?
    8.2 Basic Concepts of DST: The Geometry of Change
        8.2.1 Trajectories in state space: Predators and prey
        8.2.2 Bifurcation diagrams and chaos
        8.2.3 Embodied networks as coupled dynamical systems
    8.3 Using Dynamical Systems Tools to Analyze Networks
        8.3.1 Discovering limit cycles in network controllers for robotic insects
        8.3.2 Discovering multiple attractors in network models of reading
   Modeling the semantic pathway
   Modeling the phonological pathway
        8.3.3 Discovering trajectories in SRNs for sentence processing
        8.3.4 Dynamical analyses of learning in networks
    8.4 Putting Chaos to Work in Networks
        8.4.1 Skarda and Freeman's model of the olfactory bulb
        8.4.2 Shifting interpretations of ambiguous displays
    8.5 Is Dynamicism a Competitor to Connectionism?
        8.5.1 Van Gelder and Port's critique of classic connectionism
        8.5.2 Two styles of modeling
        8.5.3 Mechanistic versus covering law explanations
        8.5.4 Representations: Who needs them?
    8.6 Is Dynamicism Complementary to Connectionism?
    8.7 Conclusion
Sources and Suggested Readings

9 Networks, Robots, and Artificial Life
    9.1 Robots and the Genetic Algorithm
        9.1.1 The robot as an artificial lifeform
        9.1.2 The genetic algorithm for simulated evolution
    9.2 Cellular Automata and the Synthetic Strategy
        9.2.1 Langton's vision: The synthetic strategy
        9.2.2 Emergent structures from simple beings: Cellular automata
        9.2.3 Wolfram's four classes of cellular automata
        9.2.4 Langton and 8 at the edge of chaos
    9.3 Evolution and Learning in Food-seekers
        9.3.1 Overview and study 1: Evolution without learning
        9.3.2 The Baldwin effect and study 2: Evolution with learning
    9.4 Evolution and Development in Khepera
        9.4.1 Introducing Khepera
        9.4.2 The development of phenotypes from genotypes
        9.4.3 The evolution of genotypes
        9.4.4 Embodied networks: Controlling real robots
    9.5 The Computational Neuroethology of Robots
    9.6 When Philosophers Encounter Robots
        9.6.1 No Cartesian split in embodied agents?
        9.6.2 No representations in subsumption architectures?
        9.6.3 No intentionality in robots and Chinese rooms?
        9.6.4 No armchair when Dennett does philosophy?
    9.7 Conclusion
Sources and Suggested Readings

10 Connectionism and the Brain
    10.1 Connectionism Meets Cognitive Neuroscience
    10.2 Four Connectionist Models of Brain Processes
            10.2.1 What/where streams in visual processing
            10.2.2 The role of the hippocampus in memory
       The basic design and functions of the hippocampal system
       Spatial Navigation in Rats
       Spatial versus declarative memory accounts
       Declarative memory in humans and monkeys
            10.2.3 Simulating dyslexia in network models of reading
       Double dissociations in dyslexia
       Modeling deep dyslexia
       Modeling surface dyslexia
       Two pathways versus dual routes
            10.2.4 The computational power of modular structure in neocortex.
    10.3 The Neural Implausibility of Many Connectionist Models
        10.3.1 Biologically implausible aspects of connectionist networks
        10.3.2 How important is neurophysiological plausibility?
    10.4 Wither Connectionism?
Sources and Suggested Readings

Appendix A Notation

Appendix B Glossary


Name Index

Subject Index