The neural network object in the Orber o n environment is a Back-Propagation
Simulator. The simulator was designed and implemented to allow the parameters involved in the back-propagation (BP) algorithm [92] to be experimented with. The simulator allows the user to build feed forward networks, train and interrogate the networks using the back-propagation algorithm. It also allows the user to view weights, output values, bias and error values associated with the network, and finally to load and save networks. Figure 5.8 shows two neural network shells within the ORBERON environment.
The structural and functional nature of neural networks make them ideal for construction using object-oriented techniques. This is because the definition of a neural network is modular and therefore suited to object-oriented representation. A neural network can be defined as a collection of neurons that are often formed into layers (input, hidden and output), and these layers themselves are interconnected. These neurons often
Chapter 5 The Orberon Environm ent 82
all perform the same simple operations on the weight values that feed into them, to calculate the output for that neuron. Any particular learning algorithm can be applied/implemented ontop of this general structure.
HeterojeneousEnvji^^ _______________XOR_______________
INTDatOCHTE TWE («T U M St
Mould you Ilk# to lnt#rrof#t# th# network with D#t# fro* # fl 1# ... [f] Enter date .C#l Vou^ Choio#.. ...q }
[ ] ( Quit
( Expert S fstem Shell E ditor (vi)
Wofit Tfentf Andtysi» Netirsl N«iwort; WndovT Profit Trend Analyiii
--- wïfïï-ng'tgngwr-UEICKT8 * TWEEN LAYERS
1 Hidd 1 HlcM i Hldd 1 Hldd 1 Hidd 1 Hldd 1 Hidd 1 Hldd 1 Output 110 -O.OM ♦0,544 1.225 ♦l.T?# ♦0.516 -3.063 ♦0.550 ♦1.563 -0.254
I U 210 ♦O.lBl ♦0.475 -1.532 ♦l.Md ♦O.S» -1.606 ♦0.704 ♦Z.OSO -0.720 M 310 ♦0.366 "0.026 -0.667 ♦1.367 ♦0.626 -1.706 ♦0.533 ♦1.471 ♦3.361 ! I 410 ♦0.034 ♦O.Sa? -0.531 ♦l.OOS ♦0.550 -0.546 +0.313 ♦0.869 -4.667 : Î 510 ♦O.lOi ♦0.207 ♦O.OSZ ♦0.683 ♦0.147 -0.836 ♦0.360 +0.601 -1.387 : 6 61© ♦0.251 -0.239 ♦0.212 -0.491 +0.244 +0.179 +0.003 -0.396 +6.304 710 -0.266 +0.063 +0.319 -0.267 -0.425 +0.846 -0.279 -0.953 -1.472 610 -0.438 -0.306 +0.615 -1.594 -0.106 +1.374 -0,164 -1.566 -4.421 910 -0.364 +0.049 +1.003 -1.972 -0.953 +2.496 -0.938 -1.382 I 10(0 -0.147 -0.779 +1.769 -2.364 -0.911 +2.774 -0.996 -2.262
OUTPUT MATRIX H A S MATRIX
11-0.600 -0.620 -0.640 -0.660 -0.660 -0.720 -0.740 -0.760 -0.760 1+0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0.000 +0. NO SPACE TO SHOW ALL HAS Hit <r# U m > to Continu#.••....J
Figure 5.8 — BP Windows showing different Functions of the BP Neural Shell. This structure suggests a simple object-oriented representation for feed-forward neural networks (i.e. networks where each node in a layer is connected to each node in the next layer), where the basic component is a neuron or processing element object. Each processing element stores values, such as its bias, and output and error values that represent its current state. The processing element also contains a list of weights that feed into that element. A layer object is then simply an array of nodes and a network object is a array of layer objects.
The creation of this structure within an object-oriented language not only allows the implementation of the back-propagation to be modularised into operations for networks, layers and nodes, but also allows these operations to be placed within the objects that they operate on. This also means that once an object and its operations have been defined, e.g. a processing element, then that object and associated operations can be simply reused. Figure 5.9 shows the object-oriented structure of the BP neural network used within the BP Neural Shell.
Chapter 5 The Orberon Environm ent 83
Processing Elements
Inheritance
Pointer To
Window Object Message ProtocolInherited
BP Neural Structure
Figure 5.9 — BP Neural Network Structure.
The BP structure is derived from the Generic Interface Object so that it can inherit the standard message protocol. Within the network the layers have some operations that are universal to all layers but others that vary in their functionality depending on the type of layer. This situation is catered for by defining a generic base layer, with generic functions and then deriving each layer from this generic base layer class and overriding the methods that have different functionality. Using inheritance in this manner, different types of BP networks could be constructed for different types of problems, e.g. BP networks used for forecasting [120, 118] and others for pattern classification, as each have slightly different operational details.
The implementation details of the back-propagation neural network are based on the information found in Rumelhart and McClelland’s Parallel Distributed Processing, Volume 3 [90]. The main difference being that the algorithm is applied to the above object-oriented structures.
5.4. Summary
This chapter has described in detail the design and implementation of the object- oriented Orber o n environment. The Or beron hybrid environment takes into
consideration the control, message and data passing requirements of different information processing techniques. The fundamental elements of the environment that address these issues have been described.
C hapter 5__________________________ The OrberonEnvironm ent_________________________________ M
The integration of a neural network and expert system shell into the Orber o n
environment formed the test bed for the development of the fundamental components, communication protocol and the user interface. The design and implementation of the neural network and expert system processing objects, illustrated the object-oriented philosophy and also the procedure required to introduce new information processing techniques into the Or ber o n environment. The communication protocol for the neural
network and expert system processing objects was outlined along with the windowing system used by these two information processing techniques.
It has been shown that the object-oriented design of the Or ber o n environment
helps to meet the design, communication, execution and expandability requirements of integrated hybrid applications.