CAPÍTULO II. METODOLOGÍA
2.4. Métodos, técnicas e instrumentos de investigación
j
I
~I I
I I
I Chapter 8
I
I Network Performance
I
I
I I
8.0.1 Generalisation ResultsThe ability of the network to form general decision contours is tested for small two-variable problems. In fact this is rather a harsh test as in practice if any hyperplane were as densely populated by patterns as in the examples, a further extension to separate the patterns would probably be used. The contours were obtained by forming the extended space correlation matrix over all the patterns and inverting it to solve the standard weight equation. The first class was assigned an index of one and the second an index of zero.
The output threshold was set at 0.5.
The peanut cluster has proved a particularly difficult test for some types of network as it requires a larger number of decision line segments than the simple cluster problem.
As there was no noise in this experiment the function was not constrained in regions in which no patterns existed, and consequently used the freedom of these regions to allow itself to fit the patterned regions more accurately.
30~---~---~---~---~---~---~
o o o
25
o o o
20 +
+ +
0
0
15 +
+ + +
0 0
+
10
5
0
0 0 0 00 0 5 10 15 20 25 30
Figure 8.1: Localised Cluster: Patterns and Decision Contour.
92
Figure 8.2: Localised Cluster: Discriminant Function Values.
\
Figure 8.3: Localised Cluster: Thresholded Discriminant Function Values.
94
30
'-.../ 0 0 ~o
25
0 0
0 0
20 +
15 0
+ +
0
10 + +
0
0
5
0
<=>
0 0 00 0 5 10 15 20 25 30
Figure 8.4: Dual Cluster: Patterns and Decision Contour.
Figure 8.5: Dual Cluster: Discriminant Function Values.
96
Figure 8.6: Dual Cluster: Thresholded Discriminant Function Values.
30
"
, / 0 0 ~o25
0 0
0
20 + 0
+
15 +
+ +
0
10 + +
I
I+
0
5
0 0 0
c::::>
00 0 5 10 15 20 25 30
Figure 8.7: Peanut Cluster: Patterns and Decision Contour.
98
Figure 8.8: Peanut Cluster: Discriminant Function Values.
Figure 8.9: Peanut Cluster: Thresholded Discriminant Function Values.
100
restarted from different initial conditions. In the case of the new network the results are for the first time run as the adaption coefficient could be selected beforehand, and the local minimum problem did not exist. Thus when comparing the figures it is necessary to realise that in terms of total pattern presentation numbers the factor between the number of presentations to each network is in fact considerably greater.
Number of bits Volterra HLBP
2 15 95
3 17 265
4 62 1200
5 106 4100
6 292 20000
7 556 100000
8 1287 over 500000
If the figures in the above table are plotted for the two networks it is possible to see that the learning times for the current network rise quickly as a function of the number of input variables, whereas those for the new network rise at a lower rate. The results of the graph imply that the learning times for current networks are likely to be extremely large for complex problems with more than a few input variables.
In fact to see the Volterra results on the same scale a log plot is needed.
8.1.1 Interpolation Performance Comparison.
In this section the interpolative ability of two current network types Hidden-Layer and Ra-dial Basis Function (RBFs)[42] is compared with that of the new network. The network is trained on the exclusive OR class points (0,0), (1,0),(0,1),(1,1) until full convergence. The
x105
5 4.5 4 3.5
3 2.5
2 1.5 1
0.5
0 2 3 4 5 6
Figure 8.10: Presentations v Number of bits. HLBP solid, Volterra dashed line
7 8
I
rI
\ I
106~---.---.---r---.---.---~
105
---",.-,.-,
" ,
"
101~---~---~---L---~---~---~
2 3 4 5 6
Figure 8.11: LOG Presentations v Number of bits. HLBP solid, Volterra dashed line
7 8
r
adaption is then stopped and the inputs are varied over the full signal space. The class decision is then plotted as a point for clas's one and no point for class two. Graphs a) and b)
:x:2
x l
were provided by Niranjan[19].
x2
x l
104
c)
:x: 2
x l
The hidden layer network displays poor interpolative abilities which is to be expected as the decision region has linear boundaries. The Radial Basis Function network performs much better, producing suitably localised classes. This increase in performance is to be expected due to the relationship between RBFs and multidimensional interpolation. The new network converges to the best solution of the three, which is extremely near to the optimal solution, given no information other than at the class points. In fact, if the above is repeated for the network before it has fully converged, it displays results which are similar to those of RBF networks, that is the decision regions for a partially converged network is of a similar form to that of the fully converged RBF network. However the new network is not constrained to a set of a priori 'RBFs' and so can converge further. In fact this behaviour can be seen by following the decision contours as the network adapts.
8.1.2 Multiclass Performance
In practice it has not proved possible, in general, to use current networks for classification with more than 26 classes. In general, multiple class problems are addressed by using a series of networks. The flexibility of the decision surfaces of the new network, however, allows a single network to perform multiclass problems. One such problem is that of character recognition. The network was presented with an 8 by 8 pixel image of a character
I '
I
Font
Frenc Font
Figure 8.12: Multiple Mutliclass Problem.
Font
RU55iCAn Font
and required to output the character's position in the collating sequence. The network was capable of learning to recognise all 80 characters in the font. A second order Volterra extension was used with all terms present. The coefficients were set by the LMS algorithm.
It is also possible to increase the number of classes that a system can recognise by using the same vector expansion 'hardware' with multiple adaptive filter sections connected to it.