Opción Visualizar Matriz

In an effort to improve the cognitron, Fukushima and his group have developed a powerful paradigm called the neocognitron (Fukushima 1984, 1986, 1987). While the two paradigms share certain similarities, they also show fundamental differences arising from the evolution of Fukushima's research. Both are multilayerhierarchical networks organized like the visual cortex.

However, the neocognitron is more consistent with the visual-system model proposed by Hubel and Wiesel (1962, 1965, 1977). As a result, the neocognitron is far more powerful than the cognitron in its ability to recognize patterns despite translation, rotation, distortion, andchanges in scale. Like the cognitron, the neocognitron generally' uses self-organization for training, although a version has been described (Fukushima, Miyake, and Takayuki 1983) in which su-pervisedlearning was applied instead.The neocognitron is oriented toward modeling the human visual system. As such, it accepts two-dimensional patterns like those

In the visual cortex, cells have been found that respond to suchfeatures as lines and edges of specific orientations. In higher areas,cells respond to more complex and abstract figures, such as circles,triangles, and squares. At still higher levels, abstraction increasesuntil cells can be identified that respond to faces and intricateshapes. Generally speaking, cells in the higher areas receive inputsfrom a group of lower-level cells. Hence, they respond to a widerarea of the visual fjeld. Responses of these high-level cells is lessposition dependent and more tolerant of distortion Structure

The neocognitron has a hierarchical structure intended to simulatethe human visual system. As such, it consists of a succession of".<>

10-8). An input pattern is applied to the first layer and passed through the planes comprising later layers until it arrives at the output layer, where the recognized pattern is indicated.The structure of the neocognitron is difficult to diagram, but is conceptually simple. To emphasize its hierarchical nature (while simplifying the diagrams), a top-down analysis is used. That is, the neocognitron is shown to consist of layers. The layers are composed of sets of planes, and the planes are composed of cells.

v Layers

Each layer of the neocognitron consists of two arrays of cell planes (see Figure 10-9). An array of simple-cell planes receives the outputs from the previous layer, detects specific patterns, and then passes them to an array of complex-cell planes, where i:hey are processed to make the detected patterns less position dependent.

Planes

Within a layer, simple- and complex-cell planes exist as pairs; that is, for each simple-cell plane, there is a single complex-cell plane that processes its outputs. Each plane may be visualized as a twodimensional array of cells

Simple Cells

All cells in a given simple-cell plane respond to the same pattern. As shown in Figure 10-10, a simple-cell plane constitutes an array of cells, all of which are "tuned" to the same specific input pattern. Each simple cell is sensitive to a restricted area of the inputpattern, which is called its receptive range. For example, all cellsin the top simple plane in Figure 10-10 respond to a C. If a Coccurs in the input pattern to that layer, a cell responds if a "C" isdetected in its receptive range.Figure 1 0-1 0 shows that another plane of simple cells in that layer might respond to a 900 rotation of the C, another to a 1800rotation, and so on. If additional letters (and distorted versionsthereof) are to be detected, an additional plane is required for each.The receptiv(' ranges of the cells in each simple plane overlap tocover the entire input pattern for that layer. Each cell receivesinputs from corresponding regions of all complex planes in theprevious layer. In this way, a simple cell responds to an occurrence of its tuned pattern in any complex plane of the previous layer, aslong as it is within its receptive rang

Complex Cells

Complex cells serve to make the system less sensitive to the position of patterns in the input field.

To accomplish this. each complex cell receives the outputs of a set of simple cells from its

corresponding plane-in the same layer. These simple cells cover a contiguous region of a simple plane, called the receptive range of the complex cell. The firing of any simple cell in this region is sufficient to cause the complex cell to fire. In this way, a complex cell responds to the same pattern as the simple cells in its corresponding plane, but it is less position sensitive than anyone of them.Thus, each layer of complex cells responds to a larger region of the input pattern than did those in the preceding layer. This progressive increase in range, from layer to layer, yields the c/esired decrease in the position sensitivity of the overall system

Generalization

Each neuron in a layer near the input responds to a specific pattern in a precise location, such as an edge with a particular angular orientation at a given position. Each layer thereafter has a more abstract, less specific response, until at the output layer the complex cells respond to entire patterns, showing a high degree of insensitivity to their location, size, and orientation in the input

field. Used as a classifier, the output layer complex cell having the largest response indicates the detection of the associated pattern in the input field. Ideally, this detection is insensitive to the pattern's position, orientation, size, or other distortion

Calculation

Simple cells in the neocognitron have exactly the same characteristics described above for the cognitron and use essentiall y the same formulas to determine their outputs. These are not repeated here.The inhibitory cell produces an output that is proportional to the weighted-root mean square of its inputs. Note that the inputs to an inhibitory cell are identical to those of its associated simple cell and range over its responsive region in all complex planes. In symbo

where v = the output of an inhibitory cell i = ranges over all complex cells to which the inhibitory cell connects bi = the synaptic strength of the ith connection from a complexcell to the inhibitory cell Ui = the output of the ith complex cellThe weights bi are selected to decrease monotonically with distance from the center of the responsive region, and to have a sum of one.

Only simple cells have adjustable weights. These connect to complex cells in the previous layer by means of modifiable synaptic strengths, adjusted during the training process to produce a maximal response to a specific stimulus feature. Some of these synapses are excitatory and tend to increase the cell's output, while others are inhibitory and reduce the output.Figure 10-11 shows the complete arrangement of synapses between a simple cell and the complex cells in the preceding

layer

Figure 10-11. Connections from Complex Cells in One Layer toSimple Cells in the Next Layer

Each simple cell responds only to a set of complex cells within itsreceptive range. Also, an inhibitory cell exists that responds toexactly the same complex cells. Synaptic strengths of this inhibitory cell are not trained; they are selected so that the cell responds tothe average of all cell outputs to which it connects. The singleinhibitory synapse from the inhibitory cell to the simple cell is trained like any other synapse

Unsupervised Training

To train the neocognitron, a pattern to be recognized is applied tothe network input, and the synaptic strengths are adjusted layer bylayer, starting with the set of simple cells nearest the input.

Thesynaptic strength from each complex cell to a given simple cell isincreased if, and only if, the two conditions that follow are satisfied: (1) the complex cell is responding; (2) the simple cell is responding more strongly than any of its immediate neighbors (within its competition area).In this way, the simple cell is trained to respond most strongly tothe patterns that occur most frequently in its receptive region, acharacteristic that mirrors the development of pattern recognitionfound in experiments with cats. If no recognizable pattern is presented, the inhibitory cell serves to nrevent r~nri0nl I"vI'it'1

The mathematics of training and the ethod foe achieving latee. pee !ayee will decrease fcom input to output !aym. Finally, theIIInal inhibition are similar to those descnbed for the cogl11tron, so output layer has only one neuron per complex plane. Each such111

When a simple cell is selected to have its synaptic strengths at increased, it serves as a representative for all cells in its plane thereby causing their synaptic strengths to be increased in exactly the same pattern. In this way, all of the cells in a plane are trained to recognize the same feature and, after training, will do so regard less of its position in the field of complex cells in the preceding layer.

Supervised Training

In his earlier papers, Fukushima (Fukushima 1980; Fukushima and Miyake 1982) described the self-organizing, unsupervised training presented above. While this has produced impressive results, other experiments have been reported that use supervised training (Fukushima, Miyake, and Takayuki 1983). Here, the desired responses of each layer were chosen in advance by the experimenter. The weights were then adjusted using conventional two-layer training methods to produce the desired response. For example, the input layer was adjusted to recognize line segments in various orientations, much like the first layer of processing in the visual cortex. The layers that followed were trained to respond to successively more complex and abstract features, until at the output layer, the desired pattern was detected. While this effort produced a network that did an excellent job of recognizing handwritten Arabic numerals, the experimenters

abandoned biological plausibility, seeking only to maximize the accuracy of the system result Training Implementation

In the usual configuration, the receptive field of each neuron is increased in each successive layer.

Hence, the number of neurons

Both the cognitron and neocognitron are impressive in the accuracy with which they model the biological neural system. The fact that they produce results that mimic aspects of human learning and cognitive ability suggests that our DISCUSSION

understanding of the brain's function may be approaching a useful level.The neocognitron is complex and requires substantial computational resources; hence, it seems unlikely that it is an optimal engineering solution for today's pattern-recognition problems.Since the 1960s, however, the cost of computation has been halved every two to three years, a pattern that will probably continue for at least another decade. Although approaches that were computationally infeasible a few years ago are common today and may seem trivial in a few morc years, implementation of neocognitronmodels on a general-purpose computer is clearly inefficient. Thousandfold improvements in cost ancl performance need to be achieved by means of specialized architectures and custom VLSI

(very large-scale integration) techniques to make the neocognitron a practical solution to complex pattern-recognition problems; however, neither this nor any other artificial neural network para-digm should be rejected solely on the basis of its computational requirement