Si se ha sentido con demasiado sueño durante el día o ha tenido períodos de

The original “ANN Training from File_Graphical.vi” program was found to have a number of limitations in practice, and was replaced. Most notably, the program was written with little programming experience with LabVIEW, and as a result, it is far more complicated than necessary. This complexity made the program difficult to follow, producing debugging problems that may not have been resolved. The program required the user to enter a number of redundant parameters and included a number of superfluous calculations that slowed it down.

The new “ANN Training from File.vi” solves these problems. The program still performs the same function as its predecessor, but is faster, easier to use and more reliable. Firstly, the number of user inputs has been reduced so the user only needs to enter the training file path and the number of outputs within the file for the program to gather much of the required data. Other required inputs include the number of hidden layer neurons, what percentage of data is to be set aside for testing, the number of training iterations to use and parameters defining learning constant and momentum. There is a control over the magnitude of the initial random weights to improve convergence speed.

The first function of the program is to open the training file and separate it into input and output, and training and testing arrays. At this stage a new directory is created and the

training file copied to this location for future reference. There is also the option of using separate, pre–formatted, training and testing files to reduce the necessity to repeat this long process when re-running the program and to allow new testing data to be used. The user 1D array inputs of “Number of 1st Layer Neurons” and “Number of 2nd Layer Neurons” are read and used to create a 2D array of every combination of desired hidden layer neurons (if “Number of 2nd Layer Neurons” = 0 then a single hidden layer architecture is used).

Once data has been established, the program enters a For Loop for each of the hidden layer neuron combinations. Here the learning rate and momentum parameters are entered and the ANN weight arrays initialised using random values of a user selectable magnitude.

ANN training is completed within a While Loop, which finishes when a selected number of iterations have been completed. Here the learning rate and momentum values are progressively reduced and ANN training is completed by iteratively presenting the required data to the ANN training subVI “Train ANN with Output_0 to 1_SubVI_.vi” or “Train ANN with Output_-1 to 1_SubVI_.vi”. These two subVIs perform slightly different operations, as discussed below, and allow the user to select between two ANN models depending on the characteristics of the input data. Both of these subVIs take as input the ANN weights, previous ANN weight adjustments, input parameters, desired output and learning rate and momentum. They provide the ANN output prediction, updated ANN weights and ANN weight adjustments as output, as shown in Figure 5.12. The training While Loop updates these values iteratively, and so progressively trains the ANN model using randomly indexed data from the training array.

Another function is imbedded within the training While Loop to display the training progress on screen. After a specified number of iterations the “Use ANN_0 to 1_with Normal_SubVI_.vi” or “Use ANN_-1 to 1_with Normal_SubVI_.vi” subVI (which ever is selected) is applied to the testing data using the current ANN weights. Using this data the ANN RMS error is calculated and a graph of ANN predicted output is compared to the desired output on screen.

Once training was completed the program then exits the training While Loop and uses the final weights for further analysis and writes them to file. The input parameter importance is calculated first by replacing one input parameter with its median value through the whole testing array, and running it through “Use ANN_* to *_with Normal_SubVI_.vi” to monitor the effects of this change on the RMS error. This was repeated for all of the input parameters, and the difference between each RMS error and the RMS error of the unmodified testing data is saved as the parameter importance. This produces a 1D array of the values that represent how much the ANN predictive error will increase if that input fails. A small increase corresponds to a parameter that has little importance, while a large increase suggests a high importance.

The final function of the program is to produce the ANN predictions for the testing set (using “Use ANN_* to *_with Normal_SubVI_.vi”) and to write the results to file. The process input values, the desired outputs, the output error, the input parameter names, the input parameter importance and the input and output minimum and maximum values are written to the same file.

Further functionality was installed at a later date, to enable OLL ANN modelling. To do this, the sub-programs “Train ANN with OLL SubVI group” and “Use ANN with OLL SubVI group” were added to implement the OLL algorithm that has been described in the theory previously. Although the OLL training method is very different to backpropagation, these programs were designed to fit into the existing program with very minimal modifications.

5.4.5.1 Train ANN with Output_* to *_SubVI_.vi

The “Train ANN with Output_0 to 1_SubVI_.vi” and “Train ANN with Output_-1 to 1_SubVI_.vi” subVIs are both summarised in Figure 5.13. They accept all the data required for training, normalise the input and output parameters and present the result to a LabVIEW Formula Node. The node is embedded inside a Case Structure and contains C++ code for one and two hidden layer ANN training. The C++ code computes the entire training process of one iteration, replacing the highly complex graphical programming of “Train ANN_Graphical_SubVI_.vi” with text, greatly reducing the possibility of errors and making the program more versatile and easy to follow. The Formula Node results are passed to the subVI outputs and the ANN prediction is “un- normalised”.

The normalisation process was included within the subVI in this case to simplify the function of the higher-level programs. It is this normalisation process that is the only difference between “Train ANN with Output_0 to 1_SubVI_.vi” and “Train ANN with Output_-1 to 1_SubVI_.vi”. The former program normalises the data into the 0 to 1 range, while the later normalises the data into the -1 to 1 range. This is done to allow the user to offset the position of the “0” values depending on the type of inputs required. This is useful because if the ANN inputs contain “0” values, the weights have diminished ability to offset the neuron result in that region. By specifying a normalisation process that replicates the input function (i.e. if the inputs are in the 0-x range then normalise to 0-1, if the inputs are in the –x-x range then normalise to –1-1) it is possible to reduce this effect and improve ANN accuracy.

Figure 5.13: “Train ANN with Output_* to *_SubVI_.vi” basic functions

5.4.5.2 Use ANN_* to *_with Normal_SubVI_.vi

The “Use ANN_0 to 1_with Normal_SubVI_.vi” and “Use ANN_-1 to 1_with Normal_SubVI_.vi” are very similar to “Train ANN with Output_* to *_SubVI_.vi” except they perform the ANN prediction part of the C++ code only. As in Figure 5.14, the subVIs take as input the input array and the ANN weights and the max and min values. They normalise the input array into the 0-1 or –1-1 regions and finally pass the data to a Formula Node containing the C++ code. Separate Formula Nodes are selected depending on if the ANN of a one or two hidden layer architecture is desired, and the resultant ANN prediction is “un-normalised” and passed to the subVI output.

Figure 5.14: “Use ANN_* to * with Normal_SubVI_.vi”basic functions

5.4.5.3 Train ANN with OLL SubVI group addition

The original version of “ANN Training from file.vi” consisted of ANN training through backpropagation only, with a Case Structure utilised to determine the type of backpropagation model used as discussed above. As such, the addition of OLL training within this framework required an extra case to be constructed within the Case Structure, and is shown in Figure 5.15.

Unlike backpropagation training OLL models require data to be presented to them in a different way and can only work with one hidden layer. The existing program was programmed to either randomly index input/output patterns or index them repeatedly in sequential order, both of which are valid backpropagation ANN training methods. In contrast, OLL models require the input/output data to be sequentially run through the ANN model three times for a single weight update to be made, with different functions performed each time.

In this manner, the OLL training case comprises of three “rules”, which determine which of the three OLL training phases are taking place. Within each of these rules, the input/output data is run through the OLL training functions as discussed in the theory in previous sections. The three stages (loops) include:

1. Compiling the Aij and Bjk matrices based on each input/output set to determine new output layer weights Vopt.

2. Compiling the Auij and Buji matrices based on each input/output set, using the new output layer weights, to determine new 1st layer weights Wtest.

3. ANN testing with the new Vopt and Wtest for each input/output set to evaluate if new weights increase or decrease error. If error reduces then Wtest replaces the old 1st layer weights, weight factor (μ) is reduced and process starts again from

step 1. If error increases the old 1st layer weights are kept, μ is increased and the process start again from step 2.

Figure 5.15: “Train ANN with OLL SubVI Group” basic functions

5.4.5.4 Use ANN with OLL SubVI group addition

The internal structure of the OLL ANN is slightly different to backpropagation models, although they both consist of the same input and output variables. As such, a new case was inserted into each of the existing “Use ANN_* to *_with Normal_SubVI_.vi” Case Structures to contain the new OLL model. This was a simple process, and essentially takes a number of SubVIs from the “Use ANN with OLL SubVI group” to complete ANN predictions based on current weights, as shown in Figure 5.16. Furthermore, because the case input and output variables have identical functionality as the backpropagation cases, no changes needed to be made in this regard.