Programa de Prevención y Mitigación

To illustrate this point of view, consider a bowl in which a ball bearing is allowed to roll freely as shown in Figure 7.2.

Figure 7.2 Bowl and ball bearing: a system with a stable energy state.

Suppose we let the ball go from a point somewhere up the side of the bowl. The ball will roll back and forth and around the bowl until it comes to rest at the bottom. The physical description of what has happened may be couched in terms of the energy of the system. The energy of the system is just the potential energy of the ball and is directly related to the height of the ball above the bowl's centre; the higher the ball the greater its energy. This follows because we have to do work to push the ball up the side of the bowl and, the higher the point of release, the faster the ball moves when it initially reaches the bottom. Eventually the ball comes to rest at the bottom of the bowl where its potential energy has been dissipated as heat and sound that are lost from the system. The energy is now at a minimum since any other (necessarily higher) location of the ball is associated with some potential energy, which may be lost on allowing the bowl to reach equilibrium. To summarize: the ball-bowl system settles in an energy minimum at equilibrium when it is allowed to operate under its own dynamics. Further, this equilibrium state is the same, regardless of the initial position of the ball on the side of the bowl. The resting state is said to be stable because the system remains there after it has been reached.

There is another way of thinking about this process that ties in with our ideas about memory. It may appear a little fanciful at first but the reader should understand that we are using it as a metaphor at this stage. Thus, we suppose that the ball comes to rest in the same place each time because it "remembers" where the bottom of the bowl is. We may push the analogy further by giving the ball a co-ordinate description. Thus, its position or state at any time t is given by the three co- ordinates x(t), y(t), z(t) with respect to some cartesian reference frame that is fixed with respect to the bowl. This is written more succinctly in terms of its position vector, x(t)=(x(t), y(t), z(t)) (see Fig. 7.3). The location of the bottom of the bowl,

x_p, represents the pattern that is stored. By writing the ball's vector as the sum of x_p and a displacement x, x=x_p+ x, we may think of the ball's initial position as representing the partial knowledge or cue for recall, since it approximates to the memory x_p.

Figure 7.3 Bowl and ball bearing with state description.

If we now use a corrugated surface instead of a single depression (as in the bowl) we may store many "memories" as shown in Figure 7.4. If the ball is now started somewhere on this surface, it will eventually come to rest at the local depression that is closest to its initial starting point. That is, it evokes the stored pattern which is closest to its initial partial pattern or cue. Once again, this corresponds to an energy minimum of the system. The memories shown correspond to states x₁, x₂, x₃ where each of these is a vector.

Figure 7.4 Corrugated plane with ball bearing: several stable states.

There are therefore two complementary ways of looking at what is happening. One is to say that the system falls into an energy minimum; the other is that it stores a set of patterns and recalls that which is closest to its initial state. The key, then, to building networks that behave like this is the use of the state vector formalism. In the case of the corrugated surface this is provided by the position vector x(t) of the ball and those of the stored memories x₁, x₂,…, x_n. We may abstract this, however, to any system (including neural networks) that is to store memories.

(a) It must be completely described by a state vector v(t)=(v₁(t), v₂(t),… , v_n(t))

(b) There are a set of stable states v₁, v₂, v₁,…, v_n, which correspond to the stored patterns or memories.

(c) The system evolves in time from any arbitrary starting state v(0) to one of the stable states, which corresponds to the process of memory recall.

As discussed above, the other formalism that will prove to be useful makes use of the concept of a system energy. Abstracting this from the case of the corrugated surface we obtain the following scheme, which runs in parallel to that just described.

(a) The system must be associated with a scalar variable E(t), which we shall call the "energy" by analogy with real, physical systems, and which is a function of time.

(b) The stable states vi are associated with energy minima E_i. That is, for each i, there is a neighbourhood or basin of attraction around v_i for which E_i is the smallest energy in that neighbourhood (in the case of the corrugated surface, the basins of attraction are the indentations in the surface).

(c) The system evolves in time from any arbitrary initial energy E(0) to one of the stable states E_i with E(0)>E_i. This process corresponds to that of memory recall.

Notice that the energy of each of the equilibria E_i may differ, but each one is the lowest available locally within its basin of attraction. It is important to distinguish between this use of local energy minima to store memories, in which each minimum is as valid as any other, and the unwanted local error minima occurring during gradient descent in previous chapters. This point is discussed further in Section 7.5.3.