Planificación

Starting from suitable initial conditions, the analog-to-digital converter network finds a single optimal solution. This is due to the simple nature of the energy surface for this problem. In the TSP, the energy surface is highly convoluted-full of dips, valleys, and local minima-and there is no guarantee that a global optimal solu tion will be found, or even that the solution will be valid.

This raises serious questions about the reliability of the network and mitigated by the fact that finding global minima for NP-complete problems is an intractable problem that has not been solved in a reasonable amount of time in any other way; methods that are much slower and inherently serial produce results that are no better

Speed

The rapid computational capability of the network is a major advantage. This arises from the highly parallel nature of the convergence process. If implemented in analog electronics form, solutions seldom take more than a few network time constants. Furthermore, the convergence time changes little with the size of the problem. Contrast this with the more than exponential in-crease in processing time with conventional approaches. Singleprocessor simulations cannot take advantage of this inherently parallel architecture, but modern multiprocessor systems, such as the Connection Machine (with 65,536 processors!), hold great promise in solving previously intractable problems

Energy Function

It is not a trivial matter to find the function that maps a problem onto the general network-energy function. Existing solutions have been achieved through ingenuity and mathematical expertise, talents that are always in short supply, For certain problems, methods exist to determine the network weights in a systematic fashion; these techniques are studied in Chapter 7

Network Capacity

Tha maximum number of memories that may be stored in a Hopfield network is a current research topic. Because a network of N binary neurons can have as amny as 2N states , researchers were surprised to find that the maximum memory storage capacity was

If too many memories are stored, the networ will not s ,i I on some of them. Furthermore, it can remember things it has not ers, who had no mathematical way to determine how many memo- ers, who had no mathematical way to determine how many memo it had been conjectured that the maximum number of memories K that can be stored in a network of N neurons and recalled without error is less than eN, where e is a positive constant greater than one. While this limit is approached in some cases, in general it proved to be excessively optimistic; Hopfield (1982) showed ex perimentally that the general capacity limit was actually more like O.15N. Abu-Mostafa and St. Jacques (1985) have shown that the number of such states cannot exceed N, a result that is compatible with observations of actual systems and is as good an estimate as is available today.

Unit 3(a)

Bidirectional Associative Memories

Human memory is often associative; one thing reminds us of an other , and that , of still another.

If we allow our thoughts to wander , they move from topic to topic based on a chain of mental associations. Alternatively we can use this associative ability to recivere a lost memory. If we have forgotten where we left our glasses, we attempt to remember where we last saw them, who we were speaking to, and what we were doing, We thereby establish one end of an associative link and allow our mind to connect it to the desired memory.

The associative memories discussed in Chapter 6 are, strictly speaking: autoassociative; that is, a memory can be completed or corrected, but cannot be associated or corrected memory. This is a result of their single-layer structure, which requires the output vector to appear on the same neurons on which the input vector was applied.

The bidirectional associative memory (BAM) is heteroassociative; That is, it accepts an input on one set of neuron and produces a related, but different output vector on another set. Like the Hopfield, net, the BAM is capable of' generlizatioin producing correct outputs despite corrupted inputs.Also adaptive versions can abstract, extracting the ideal from a set of noisy example. These characteristrics are strongly reminiscent of human mental function and bring artificial neural networks one step closer to an emulation of the brain.

Recent publications (Kosko 1987a; Kosko and Guest 1987) have presented several forms of bidirectional associative memories. Like most important ideas, this one has deep roots; for example, the work of Grossberg (1982) presents several concepts that are important to BAMs. No attempt is made here to resolve questions of priority among research works; references are provided solely on the basis of their value in illuminating and expanding the subject matter.

Topic 1:BAM STRUCTURE

Figure 7-1 shows the basic BAM configuration: This format is considerably different from that used by Kosko. It was chosen to high light the similarity to the Hopfield network and to allow extension to systems with more layered. Here an input vector A is applied to the weight network W and produces a vector of neuron outputs B. Vector B is then applied to the transpose of the first weight network W which produces new output for vector of neurons A. The process is repeated until the network arrives at a stable point, when neither A nor B is changing. Note that the neurons in layers 1 and 2 operate as in other paradigms, producing the sum of the weighted inputs and applying it to the activation function F. This process may be expressed in symbols as follows:

or in vector form bi = FΣAW where

B = the vector of outputs from layer 2 A = the vector of outputs from layer I

W = the weight matrix between layers I and 2 Similarly,

A= F(BW’)

where W’ is the transpose of matrix W.

As discussed in Chapter 1, Grossberg has shown the advantages of using the familiar sigmoidal (logistic) activation function

OUTi =1/(1=exp-NET) where

OUTi = the output of neuron i

NETi = the weighted sum of the inputs to neuron i λ = a constant that determines the slope of the curve

In the simplest version of the BAM, the constant λ is made large, there by producing an activation function that approaches a simple threshold. For the time being, we shall assume that the threshold function is used.

We shall assume that there is memory within each neuron in layers 1 and 2 , and that their outputs change simultaneously with each tick of a master clock, remaining constant between ticks. Therefore, the neurons will obey the rules that follow

OUTln+ 1)= 1 if NETln) > 0 OUTln + 1) = 0 if NETln)< 0 OUT,(n + 1)::: OUT(n) if

where OUT,(n) is the value of an output at time n.

Note that as in networks described previously, layer 0 does no computation and has no memory; it serves only as a distribution point for the outputs of layer 2 to matrix W'.