This section introduces the two visual systems hypothesis, first proposed by the neurolo- gists Leslie Ungerleider and Mortimer Mishkin. The two visual systems hypothesis is important both because of the tools that were used to arrive at it (including the study of brain-damaged patients and experiments on monkeys) and because it illustrates a bottom-up, as opposed to top-down, way of studying the mind.
B O X 3 . 1 What does each lobe do?
n Frontal lobe – reasoning, planning, parts of speech, movement, emotions, and problem solving
n Parietal lobe – movement, orientation, recognition, perception of stimuli
n Occipital lobe – associated with visual processing
n Temporal lobe – associated with perception and recognition of auditory stimuli, memory, and speech
Primary visual cortex
Information from left half of visual field
Optic nerve
Information from right half of visual field Lateral geniculate nucleus Field of view of right eye Field of view of left eye
Figure 3.4 The primary visual pathway. Note the contralateral organization, with information from the right side of space processed by the left side of the brain.
Ungerleider and Mishkin suggested that visual information does not take a single route from the primary visual cortex. Instead, the route the information takes depends upon the type of information it is. Information relevant to recognizing and identifying objects follows a ventral route (see Box 3.2) from the primary visual cortex to the temporal lobe, while information relevant to locating objects in space follows a dorsal route from the primary visual cortex to the posterior parietal lobe. The two routes are illustrated inFigure 3.5
The reasoning that led Ungerleider and Mishkin to this conclusion came both from the study of cognitive impairments due to brain damage and from neuroana- tomical experiments on monkeys. The neuroanatomical experiments were their distinctive contribution. By the time Ungerleider and Mishkin were writing there was already considerable evidence from brain-damaged patients that damage to the temporal and parietal lobes produced very different types of cognitive problem. Damage to the temporal cortex is associated with problems in identifying and recog- nizing objects, while damage to the parietal cortex tends to result in problems locat- ing objects.
Evidence of this type has always been very important in working out the function of the different lobes (seeBox 3.1for a standard“division of labor” between the lobes). But being able to localize specific functions in this way falls a long way short of telling us the full story about the path that information takes in the brain. For this Ungerleider and Mishkin turned to experiments on monkeys.
The particular type of experiments that they carried out are called cross-lesion discon- nection experiments. This is a methodology explicitly designed to trace the connections between cortical areas and so to uncover the pathways along which information flows. It addresses a fundamental problem with making inferences about the function and spe- cialization of particular brain areas from what happens when those areas are damaged. Simply finding specific cognitive problems associated with damage to a specific brain region gives us no way of telling whether the impaired cognitive abilities are normally carried out by the damaged brain region itself, or by some other brain region that
B O X 3 . 2 Brain vocabulary
Neuroscientists and neuroanatomists use an unusual vocabulary for talking about the layout of the brain:
Rostral = at the front Caudal = at the back Ventral = at the bottom Dorsal = at the top Ipsilateral = same side Contralateral = opposite side
crucially depends upon input from the damaged brain region. Solving this problem cannot be done simply by observing the results of brain damage. Precise surgical inter- vention is required, in the form of targeted removal of specific brain areas to uncover the connections between them.
The cross-lesion disconnection experiments exploit the fact that the cerebrum is divided into two hemispheres, with duplication of the principal cortical areas. Suppose that investigators think that they have identified a cortical pathway that connects two cortical areas. They can remove the area assumed to be earlier in the pathway from one hemisphere and the area assumed to be later from the other hemisphere. Ungerleider and Mishkin, for example, working on the hypothesis that there is a pathway connecting the primary visual cortex and the inferior temporal area, per- formed surgery in monkeys to remove the primary visual cortex from one hemisphere in monkeys and the inferior temporal area from the other hemisphere. This destroyed the postulated pathway in each hemisphere. However, because the hemispheres can communicate through a large bundle of fibers known as the corpus callosum (illus- trated inFigure 3.2), it turned out that there was little or no loss of function in the monkeys.
So, for example, it is well documented that monkeys who have had their inferior temporal cortex removed from both hemispheres are severely impaired on basic pattern discrimination tasks. But these pattern discrimination tasks were successfully performed by monkeys with primary visual cortex removed from one hemisphere and inferior temporal cortex from the other. Cutting the corpus callosum, however, reduced performance on those pattern discrimination tasks to chance and the monkeys were unable to relearn it. Using experiments such as these (in addition to other types of neurophysiological evidence), Ungerleider and Mishkin
Dorsal stream
Ventral stream
Figure 3.5 Image showing ventral stream (purple) and dorsal stream (green) in the human brain visual system.
conjectured that information relevant to object identification and recognition flows from the primary visual cortex to the inferior temporal cortex via areas in the occipital lobe collectively known as the prestriate cortex. They called this the ventral pathway.
Ungerleider and Mishkin identified a completely different pathway (the dorsal path- way) leading from the primary visual cortex to the posterior parietal lobe. Once again they used cross-lesion disconnection experiments. In this case the task was the so-called landmark task, illustrated in the top left part ofFigure 3.6
In the landmark task monkeys are trained to choose food from one of two covered foodwells, depending on its proximity to a striped cylinder. The striped cyclinder is moved at random and what the task tests is the monkey’s ability to represent the spatial relation between the striped cylinder and the two foodwells.
The basic methodology of the experiments was the same as for the visual recognition pathway. The surgery proceeded in three stages. In the first stage (B in Figure 3.6) the posterior parietal cortex was removed from one side. The second stage (C) removed the primary visual cortex on the opposite side. The final stage (D) was a transection (severing) of the corpus callosum.
As indicated inFigure 3.6, the monkeys were tested on the landmark task both before and after each stage. However, the impairments on the landmark task were much more complicated than in the earlier experiments. The numbers in Figure 3.6 indicate the number of trials required to train the monkeys to a 90 percent success rate on the landmark task. So, for example, prior to the first stage of the surgery the average number of training trials required was 10. After lesion of the posterior parietal cortex the number of training trials went up to 130.
One interesting feature of these experiments is that the most severe impairment was caused by the second stage in the surgery, the removal of the primary visual cortex (in contrast to the other experiments on the visual recognition pathway, where severe impairments appeared only with the cutting of the corpus callosum). Ungerleider and Mishkin concluded from this that the posterior parietal cortex in a given hemisphere does not depend much upon information about the ipsilateral visual field (seeBox 3.2) from the opposite hemisphere’s primary visual cortex.
This raises the following intriguing possibility, since it is known that each hemisphere is specialized for the contralateral region of space. It may be that the posterior parietal cortex in each hemisphere is specialized for processing information about the opposite region of space. This would mean, for example, that the left posterior parietal cortex processes information about the layout of space on the perceiver’s right-hand side. This could be particularly important for thinking about the neurological disorder of unilat- eral spatial neglect. Patients with this disorder typically“neglect” one half of the space around them, eating food from only one side of the plate and describing themselves as unaware of stimuli in the neglected half of space. Unilateral spatial neglect typically follows damage to the posterior parietal cortex in one hemisphere (most often the right) and the neglected region is contralateral to the damage (so that, most often, the left-hand side of space is neglected).
5 cm 20 cm Pre-op trials 10 Post-op trials 130 Foodwell Landmark Foodwell (a) (b) Pre-op trials 70 Post-op trials 880 Pre-op trials 30 Post-op trials 400 (c) (d) ?
Figure 3.6 Design and results of Ungerleider and Mishkin’s cross-lesion disconnection studies. (a) Landmark task. Monkeys were rewarded for choosing the covered foodwell located closer to a striped cylinder (the “landmark”), which was positioned on the left or the right randomly from trail to trail, but always 5 cm from one foodwell and 20 cm from the other. Training was given for 30 trials per day to a criterion of 90 correct responses in 100 consecutive trials. (b) Discrimination retention before and after first-stage lesion (unilateral posterior parietal; V = 3); 10 preoperative trials and 130 postoperative trials. (c) Discrimination retention before and after second-stage lesion (contralateral striate; y = 3); 70 preoperative and 880 postoperative trials. (d) Discrimination retention before and after third-stage lesion (corpus callosum; N = 3); 30 preoperative and 400 postoperative trials. At each stage the lesion is shown in dark brown and the lesions of prior stages in light brown. Arrows denote hypothetical connections left intact by lesions. (Adapted from Ungerleider and Mishkin1982)
The visual systems hypothesis was a very important step in mapping out the connect- ivityof the brain. Ungerleider and Mishkin’s basic distinction between the “what” system (served by the ventral pathway) and the“where” system (served by the dorsal pathway) has been refined and modified by many researchers (see the references in the further reading section of this chapter). However, the idea that there is no single pathway specialized for processing visual information, but instead that visual information takes different processing routes depending upon what type of information it is, has proved very enduring. From the perspective of cognitive science, the significance of the two visual systems hypothesis is that it exemplifies in a particularly clear way the bottom-up study of how information is processed in the mind.
There are recognizable affinities between what Ungerleider and Mishkin were doing, on the one hand, and the top-down approach of cognitive scientists such as Marr, on the other. So, for example, both are concerned with identifying distinct processing systems in terms of the functions that they perform. The real difference comes, however, with how they arrive at their functional analyses. For Marr, the primary driver is top-down think- ing about the role of visual processing within the overall organization of cognition and the behavior of the organism. For Ungerleider and Mishkin, the primary driver is think- ing that starts at what Marr would term the implementational level. Instead of abstract- ing away from details of the channels and pathways between neural systems along which information processing flows, Ungerleider and Mishkin started with those chan- nels and pathways and worked upwards to identifying distinct cognitive systems carry- ing out distinct cognitive functions.
Exercise 3.3 Make as detailed a list as you can of similarities and differences between these two different approaches to studying the organization of the mind.
3.3
Extending computational modeling to the brain
Computational modeling is one of the principal tools that cognitive scientists have for studying the mind. One of the best ways to understand particular cognitive abilities and how they fit together is by constructing models that“fit” the data. The data can take many different forms. In the case of SHRDLU, the data are given simply by the human ability to use language as a tool for interacting with the world. In other models, such as the two visual systems hypothesis considered in the previous section, the data are experimentally derived. The two visual systems hypothesis is essentially a model of the visual system designed to fit a very complex set of neurological and neurophysiological data. Experiments on mental rotation and mental scanning provide the data for the model of mental imagery proposed by Kosslyn and others.
All of the models that we have looked at in our historical survey share certain very basic features. They all think of cognition in terms of information-processing mechan- isms. Whereas Ungerleider and Mishkin were interested primarily in the neural pathways and channels along which information travels, the other models we have considered
have focused primarily on the algorithms that govern information processing. Loosely speaking, these algorithms have all been driven by the computer model of the mind. They have all assumed that the processes by which information is transformed and transmitted in the brain share certain general characteristics with how information is transformed and transmitted in digital computers. And just as we can study computer algorithms without thinking about the hardware and circuitry on which they run, so too do most of these models abstract away from the details of neural machinery in thinking about the algorithms of cognition.
There are several reasons, however, why one might think that abstracting away from neural machinery in studying the algorithms of cognition may not be a good idea. One set of reasons derives from the temporal dimension of cognition. Cognitive activity needs to be coordinated with behavior and adjusted on-line in response to perceptual input. The control of action and responsiveness to the environment requires cognitive systems with an exquisite sense of timing. The right answer is no use if it comes at the wrong time. Suppose, for example, that we are thinking about how to model the way the visual system solves problems of predator detection. In specifying the information- processing task we need to think about the level of accuracy required. It is clear that we will be very concerned about false negatives (i.e. thinking that something is not a predator when it is), but how concerned should we be about false positives (i.e. thinking that something is a predator when it is not)?
Exercise 3.4 Can you think of a cognitive task for which it is more important to minimize false positives, rather than false negatives?
There is a difference between a model that is designed never to deliver either false positives or false negatives and one that is designed simply to avoid false negatives. But which model do we want? It is hard to see how we could decide without experimenting with different algorithms and seeing how they cope with the appropriate temporal constraints. The ideal would be a system that minimizes both false negatives and false positives, but we need to factor in the time taken by the whole operation. It may well be that the algorithm that would reliably track predators would take too long, so that we need to make do with an algorithm that merely minimizes false negatives. But how can we calculate whether it would take too long or not? We will not be able to do this without thinking about how the algorithm might be physically implemented, since the physical implementation will be the principal determiner of the overall speed of the computation. Moreover, the mind is not a static phenomenon. Cognitive abilities and skills them- selves evolve over time, developing out of more primitive abilities and giving rise to further cognitive abilities. Eventually they deteriorate and, for many of us, gradually fade out of existence. In some unfortunate cases they are drastically altered as a result of traumatic damage. This means that an account of the mind must be compatible with plausible accounts of how cognitive abilities emerge. It must be compatible with what we know about how cognitive abilities deteriorate. It must be compatible with what we know about the relation between damage to the brain and cognitive impairment.
All of these factors derive directly from the fact that minds are realized in brains. We know, for example, that cognitive abilities tend to degrade gracefully. Cognitive phenom- ena are not all-or-nothing phenomena. They exhibit gradual deterioration in perform- ance over time. As we get older reaction times increase, motor responses slow down, and recall starts to become more problematic. But these abilities do not (except as a result of trauma or disease) disappear suddenly. The deterioration is gradual, incremental, and usually imperceptible within small time frames. This type of graceful degradation is a function of how brains are wired, and of the biochemistry of individual neurons. The same holds for how cognitive abilities emerge and develop. Brains learn the way they do because of how they are constructed – and in particular because of the patterns of connectivity existing at each level of neural organization (between neurons, populations of neurons, neural systems, neural columns, and so forth). It is plausible to expect our higher-level theories of cognitive abilities to be constrained by our understanding of the neural mechanisms of learning.
Exercise 3.5 Can you think of other reasons for thinking that we should not theorize about cognition without theorizing about the brain?
A new set of algorithms: Rumelhart, McClelland, and
the PDP Research Group, Parallel Distributed Processing:
Explorations in the Microstructure of Cognition (1986)
The very influential two-volume collection of papers published by Rumelhart, McClelland, and the PDP research group in 1986 proposed and pursued a new set of abstract mathematical tools for modeling cognitive processes. These models, sometimes called connectionist networks and sometimes artificial neural networks, abstract away from many biological details of neural functioning in the hope of capturing some of the crucial general principles governing the way the brain works. Most artificial neural networks are not biologically plausible in anything but the most general sense. What makes them so significant, however, is that they give cognitive scientists a bridge between algorithm and implementation.
We will be looking in much more detail at artificial neural networks in later chapters (particularlyChapters 8and9). For the moment we will simply give a brief sketch of some of the key features. The first is that they involve parallel processing. An artificial neural network contains a large number of units (which might be thought of as artificial neurons). Each unit has a varying level of activation, typically represented by a real number between 1 and 1. The units are organized into layers with the activation value of a given layer determined by the activation values of all the individual units. The simultaneous activation of these units, and the consequent spread of activation through the layers of the network, governs how information is processed within the network. The processing is parallel because the flow of