The purpose of network training is to assign each of the ANN weights with a unique real number to enable the network to perform the transform that yields the required outputs with maximum accuracy. This means that every ANN must be programmed with different weights for different applications and, because it is impossible to compute the weights directly, the network must be trained. This involves presenting the network with a set of measured data from the system it is to model, which it then uses to assign its own values to the weighting factors and is referred to as “supervised training”. The way it does this depends on the network type, the acquired training data and the learning rules it uses, but is always a repetitive and iterative process that can be very time-consuming.
In order to produce both good performance and a reasonable learning rate from the ANN, a number of factors must be considered. In particular, the learning rules used can have a great effect on how quickly the ANN learns the process. The training data must offer an adequate representation all of the operating conditions of the system to be modelled, as well as being presented to the network in a particular way. Network architecture is also an important consideration. The number of hidden layers, the number of neurons within them and the way they are interconnected greatly effects network performance. [74, 96, 99]
Firstly, it is important that training data sufficiently represents all operational aspects of the system to be modelled. In particular, the dynamics of a system normally contains a number of input subgroups that have their own tendency towards a particular output prediction. As such, each subgroup must be adequately represented within the training set to allow ANN training of the complete system. Where noise is present within a system, each subgroup must contain enough data within it to include the effects of statistical variation of the process.
It is important to ensure that the order in which each subgroup is presented to the system is spread out. If the network is trained with just one example at a time (called a “pattern”) in the order that they were measured, the weights set meticulously for one fact could be drastically altered in the learning of another. In short, it may forget previous lessons when learning something new. This is undesirable, and the training set should ensure that the ANN learns everything together so it assigns weights that suit the entire
system. When learning off-line this is normally accomplished by randomising the order of training patterns, replacing the time series of data with a randomised series.
In addition to verifying that the training data is sufficiently represented, network performance can usually be improved by normalising the training data to ensure each input has similar magnitudes. This makes sure that the network is not biased towards inputs that are of a higher magnitude than others in the training set, which can create training problems. Normalising the output is also an important step towards improving network performance, since most training algorithms attempt to minimise the total error of the outputs. Using data that is not normalised will cause the network to train the output with the largest magnitude (and thus statistically the largest error) to be as accurate as possible, to the exclusion of the accuracy of other outputs.
The black box nature of the ANN models means that, once trained, their predictive performances must be observed to obtain estimates of their accuracy. In particular, the network could have made a number of generalisations within the test data that are not supported in reality. It is therefore important to gather a second set of data to be run through the ANN so that a comparison can be made between the desired output and the actual output. This is referred to as the “testing data”, and if the network cannot produce the desired accuracy using this data it may have to be redesigned or the training set may need to be broadened.
One important consideration is the number of internal neurons - too few will starve the network of the resources it needs, while too many will increase the training time and could cause overfitting. Overfitting can be a particular problem because it causes the network to memorise the training data, rather than generalise it, as shown below in Figure 3.6. In this case, the graph on the centre shows a good generalised fit to the somewhat noisy training data, while the graph on the right has created a curve that fits all of the training data very well but does not reflect the true data relationship, and the graph on the left has not learnt the process well. This also highlights one of the principle benefits of utilising a completely new set of data for ANN testing, because this should clearly show that the training generalisations are not sufficient, or if overfitting has occurred.
Figure 3.6: Effect of overfitting [96]
The number of input layer neurons can affect the accuracy of the network. In particular, the addition of input parameters that have little or no influence on the system outputs can significantly increase the network error because the ANN is forced to waste resources trying to identify relationships that are not there. In a similar manner, over-representing specific input parameters within the model can reduce accuracy. In this case using, say, engine crank speed and engine cam speed as ANN inputs will confuse the training process, because these parameters are related (crank speed = k * cam speed). It is therefore important to identify the minimum number of inputs required to successfully model the system for optimum performance [96, 99].
Therefore, the most appropriate ANN model of a process can be determined by adding training data as needed, iteratively altering the internal architectures of the neuron layers and iteratively removing or adding appropriate input parameters. This represents significant investigation, especially when considering that the training times for large ANNs can be in the order of days or weeks. As such, after an ANN has shown its capacity to model a system within reasonable error bounds, the process of finding the best ANN model can be very time consuming. However, once the best architecture is identified, ANN modelling becomes very simple and usually requires less processing power and time than traditional mathematical models. In the field of automotive technology the potential benefits that this can offer are large, and many investigations have been completed into different applications.