Arquitectura Bioclimática

N eaL in Reinforcement P ercentage difference 6 0 -r 3 0 -L S in g le B roken SLTfœe B to k e n s u r f œ e

B d lP a k R einforcement P ercen tage d fferen ce

S in g le B to k e n S L rfcre

B to k e n

s u f a c e

M ecn S e e n Level P ercen tage dfferen ce

EHT V d ta g e Level S tc n d a d Deviation P ercen tage d fferen ce

s in g le B to k m su rface B to k e n s u rfa c e s u rfa c e 100 - - T w o B ro k en to n e s u rfa c e s u rfa c e

H Input Networks Only Q Continuous Output N et ® N et/A dcptive Registers

Figure 8.28. A comparison of "best of each" reinforcement results from the experiments in this chapter with comparisons in terms of percentage difference with respect to the performance of the basic component system from chapter 5. All results are taken from the broken-surface test sets (surface 3).

This chapter has presented an empirical exploration of some implementational aspects of leaming and adaptivity in the sub-behavioural mechanisms of a behaviour based control structure. More specifically a number of additions to the basic APSM building blocks of the subsumption architecture were made in an effort to provide a degree of automatic configuring and continuous adaptation to the ongoing environmental situation. Firstly the addition of a simple feedforward artificial neural network to the input-state identification part of an AFSM was tested and then followed by the addition of an automatic adaptation of the AFSM output registers. Finally these two mechanisms were combined and tested against a third method for providing adaptivity in an AFSM process, the continuous-valued output counterpropagation network. Figure 8.28 summarises the results by comparing the performances of the various implementations in terms of reinforcement values for individual AFSM processes and also at a system level in terms of mean

scan-level and the standard deviation of the EHT voltage output. The results for the broken- surface training examples are shown and the performance of each configuration on the other main test surfaces is compared. It can be seen that a clear improvement has been achieved through the addition of these mechanisms. In fact, an extra aspect not emphasised by the graphs in figure 8.28 is that of the improved robustness and resistance to noise and reduced inputs, as evidenced in figure 8.15 and figure 8.16.

In the last section the performance of the continuous-valued output counterpropagation, in terms of mean reinforcement values, did not appear greatly impressive. However, given the comparative graphs in figure 8.28 that show more detail in terms of a system-level examination using mean scan-level and the standard deviation of the EHT voltage output, the results are more notable. In the final analysis the choice of mechanisms is probably one that is application-specific and there is certainly nothing to preclude the use of both techniques at different points in the same control structure. It may be that in the long term the "pure" network approach will provide more scope for further research through the use of more advanced artificial neural network techniques. But it may also be beneficial to keep the adaptive output function, which is not very expensive in computational terms, and use this to adapt the basic AFSM rule which is in turn used to update the feedforward network as before. The adaptation then becomes a background mechanism only, manifesting itself through producing changes in behaviour via its influence on modification of the network weights. Future work of this nature is discussed in more detail in the next chapter.

Multi-Agent Interactions

One of the original aims of this work was to explore the feasibility of embedding leaming and adaptation mechanisms into the functionality of a low-level behaviour based control process. The subsumption architecture was used as a target for the research in part because it provided a framework of asynchronous control processes, the augmented finite state machines. The issues of synchronisation have been largely left out of the discussion so far, but one or two factors have materialised as being important in this respect. These concern firstly the nature of setting up the parameters of the adaptation and leaming mechanism so that the inter-AFSM processes will interact in a sustainably stable manner. The second factor concems any explicit synchronisation strategy that the designer builds into the system. For example, a handshake type of signal between two AFSMs (which was included in the user interface layer described in chapter 5). It was the case with the BallPark and NearLin AFSMs in the photomultiplier control stmcture in this chapter that it was not possible to guarantee to maintain any interaction built in by the designer as part of the AFSM basic mle once the network had taken over the state identification/action selection process. Consequently the design and initialisation of an enhanced/adaptive AFSM process had to be such that any change in interaction could be compensated for elsewhere within the distributed system. Each component must be able to cope with continually changing characteristics of any interaction of processes. The problem is that the design task now includes

the local initialisation of internal process adaptation mechanisms. The agent-community-oriented view of systems at this level consequently becomes more relevant. We therefore take up this view as a topic of discussion in both the next and final chapters.