Servicios de Red inteligente, Tarificación adicional y Números cortos

Artificial neural networks or simply neural networks [47] are biological inspired mathematical structures that are used to estimate a function between input and

ALGORITHMS FOR LEARNING AUTOMATION

output variables. These neural networks have a functional unit called a “neuron” that is repeated in a layer, with multiple layers and are interconnected using weights to form a network. Fig. 3.1 shows a typical neural network with two hidden layers. In the figure, the circles seen are the neurons where each neuron from one layer is connected to each layer of the next layer. The arrows represents the weights that the output of each neurons get multiplied with and act as inputs for the succeeding layer of neurons. By knowing all the optimum weights, a relation between input and output can thus be calculated.

The neural networks use a supervised learning algorithm like a back propagation algorithm. It is called as supervised because while using the neural networks there is a target data sample available for every input data sample. Once a prediction is made by the network, the error (typically root mean squared error) is calculated as the difference between the predicted output and the target output. The objective is thus to define “optimal” weights that minimize this error for the data set used for training. The gradient of the error function is calculated and the error is back propagated through the layers to change the weights in the gradient decent direction and the weights are updated. This updates the predicted value to reduce the error with the target values. Thus the target data sample “supervise” the prediction made by the neural network and guide the prediction. In contrast, as discussed in the next section, reinforcement learning does not have a target data but only a feedback quantifying how good an action performed by agent is. There is no information providing what the action should be. The algorithm finds the best action over time for a given state of the system to move into a new state.

Even though neural networks are widely used for pattern recognition, classifica- tion, regression, they are not efficient in decoding pattern for a time based series of events. This is due to the fact that neural networks have no “memory” to remem-

ALGORITHMS FOR LEARNING AUTOMATION

Figure 3.1: An Artificial Neural Network with two hidden layers.

ber the previous output and use it to impact the next output. Even though the inputs from previous times can be stacked to make a multi-dimensional input, the information about the last prediction cannot be known for the current prediction. This is essentially required for a time series data where decisions are based on earlier decisions.

With respect to the task of picking and placing for the work presented here, the subtask such as picking up the ring cannot take place if the previous task predicted was placing the ring on the target peg. To place the ring, the ring should already have been picked. However, the neural network does not have the information of the previous predicted value and thus could predict “picking up ring” after “placing the ring” was already predicted. This is very undesirable and would result in a low accuracy model.

ALGORITHMS FOR LEARNING AUTOMATION

Figure 3.2: A simple recurrent neural network (RNN) [49]. The circles represent the repeating module of the network and is generally a simple tanh activation function in simple recurrent neural networks.

RNNs, the neurons are interconnected with each other to form loops and therefore can preserve an internal state which can be used to process the next output. Due to this feature, RNNs can be used for temporal dynamic behaviors and can be used in tasks like continuous handwriting recognition [50] and speech recognition [51], [52]. [53] proved that RNNs can successfully be used to generate any trajectories for a n-dimensional dynamical system. Fig. 3.2 shows a simple RNN. In the figure, the circles represent a neural network and not a single neuron.

Long Short Term Memory networks (LSTMs) first introduced by Hochreiter and Schmidhuber (1997) et. al. [54] are RNNs with some additional features and are capable of learning long term temporal dependencies. The principal behind the LSTMs are that they have different gates that modify the internal cell state. These gates are simply a combination of neurons of the neural network. Fig. 3.3 shows the structure of the LSTM networks. From this figure it can be seen that LSTM has four layers or gates of neurons performing specific functions.

The workings of the LSTM primarily depend upon the cell state that is the horizontal line at the top of the cell. This cell state carries the temporal information and is changed and used by the different gates of the LSTM. The information carried in the cell state at time t is denoted by Ct. The first layer in the LSTM network

ALGORITHMS FOR LEARNING AUTOMATION

Figure 3.3: The repeating module of the LSTM network [55] containing four layers of neurons.

is known as the “forget layer” where based on the input and last predicted output, the sigmoid function calculates a number ft between 0 and 1 and is multiplied by the cell state. This decides how much of the information of the cell state is kept with zero representing none and 1 representing all.

ft= σ(wf ∗ [xt, ht−1] + bf) (3.1)

where σ(x) = _1+eexx, wf is the weights associated with the forget layer and bf is the bias.

The second layer is called the “input layer” where the cell state is updated based on the current inputs. Another sigmoid neuron is used to compute a number itthat decides the amount the current input affects the cell state. This itis then multiplied by the input information normalized by the hyperbolic tangent function ˆCt.

it= σ(wi∗ [xt, ht−1] + bi) (3.2)

ˆ

ALGORITHMS FOR LEARNING AUTOMATION

where tanh(x) = e_exx−e_+e−x−x and w∗and b∗are the weights and biases of the respective layers. With these two layers defined, the cell state, Ct can now be calculated for time t as,

Ct = it∗ ˆCt+ ft∗ Ct−1 (3.4)

The final layer known as the “output layer” where the output for the current time is computed using the cell. However, before the output is calculated, the parts of cell state to use is calculate by another sigmoid layer. This is represented by ot and can be calculated as,

ot= σ(wo∗ [xt, ht−1] + bo) (3.5)

Finally the output is given by,

ht= ot∗ tanh(Ct) (3.6)

This process is repeated recursively for all time steps for a given training data and the optimized weights and biases are calculated as for a regular neural network.