Modelo de convenio

Sensory - Motor Architecture

Figure 6.4 below illustrates the neural architecture of the virtual insect using heterosynaptic plasticity and excitability modulation:

Figure 6.4: Neuro-inspired controller of the virtual insect with the proposed heterosynaptic implementation.

Excitability Modulatory Subcircuit

Compared to the previous circuit in Figure 6.3 the topology of the sub-network made up from PD neurons and receptors is still the same in Figure 6.4. The change in the dynamics of the system is mostly introduced by the modulatory effect of the neurons EM and PM. The neuron EM is an excitability modulatory neuron which is connected to motoneurons R and M through inhibitory synapses. The activation of EM leads to the transmission of a modulatory signal that decreases the concentration of the EMS in the postsynaptic neurons R and M. The amplitude of the excitability-inhibition effect is given by the weights WEM r

and WEM mfor neurons R and M respectively. During the strong inhibition effect the motoneurons become significantly unresponsive to the pulses coming (from PD neurons) through synapses with affinity to the EMS. Given that the synapses with EMS affinity (illustrated with dashed lines in Figure 6.4) originate from the afferent neurons A, B and C, it implies that the perceived visual stimuli will have a weakened effect (or not effect at all) on the motor behaviour during a certain period of time after the activation of the modulatory neuron EM.

The rationale for the unresponsive motor interval is to allow the artificial insect to ex- plore its surrounding environment for a short period of time without the bias of already learned conditioned stimulus-response behaviour. This should allow the insect to unlearn already acquired behaviour while learning to associate new stimulus-reflex behavioural pairs. As shown in Figure 6.4, neuron EM has an incoming synapse from PD neuron H1 which delivers periodic pulses acting as a hearthbeat. Because of the low synaptic weight WH1emfrom H1 to EM, the activation of EM requires the arrival of several pulses during a long period of time in order to occur. In this way, the activation of EM is triggered periodically by H1 unless EM receives inputs from other incoming synapses. Neuron EM has an incoming inhibitory synapse from receptor F. This means that any encounter with a rewarding stimulus will disrupt or possibly reset the otherwise periodic activation behaviour of neuron EM.

The function of neuron EM including its incoming and outgoing synapses is to allow the system to change its behaviour in the absence of rewarding stimuli during a given period of time. Thus, for this experimental setup, the modulatory subsystem works as a "take the risk or starve" mechanism that temporarily allows the artificial insect to explore its surrounding world ignoring the previously learnt associations. This mechanism is inspired by the behaviour observed in nature where animals under extreme survival conditions (for instance, starvation, escaping from a predator or competing for mating opportunities) will try taking again certain routes that have already been associated with danger or other aversive responses. In other words, they give a second chance to an already failed or dangerous solution.

Plasticity Modulatory Subcircuit

The neuron PM is a plasticity modulatory neuron which is connected to motoneurons R and M through excitatory synapses. The activation of PM leads to the transmission of a modulatory signal that increases the concentration of the PMS in the postsynaptic neurons R and M. As shown in Table 6.1 the equilibrium concentration of PMS in motoneurons R and M is set at a very low value in order to ensure that plasticity in both neurons is almost neglected in the absence of a plasticity modulatory signal. As shown in Figure 6.4, neuron PM has incoming excitatory synapses from receptors P and F. Hence, any noxious or rewarding stimulus will trigger the activation of the PM neuron and consequently allow plasticity in the motoneurons R and M.

From the point of view of the behaviour of the artificial insect, the reason for the low equilibrium concentration of PMS in R and M is to allow the insect to only learn the association between noxious and rewarding stimuli with their corresponding elicited reflexes when the reflex action is triggered by a receptor associated with an unconditioned stimulus input (i.e. nociceptive or rewarding) and not by previously conditioned neurons (for instance, visual afferent neurons). This is to prevent the positive plasticity feedback loop that emerges when the activation of the afferent neurons (A, B or C) reinforces their own synapses producing runaway dynamics in the long term.

In order for neuron PM to elicit plasticity in the postsynaptic motoneurons R and M, it is necessary that the modulatory signal arrives at R and M before they become activated (initiate an action potential). This is because in a PD neuron the STDP processing of spikes with pre-before-post timing occurs during the initiation of the action potential (see InitiateActionPotential component in figure 5.10 of chapter 5). Otherwise, if the modulatory signal arrives just after the activation of the postsynaptic neuron, then all the incoming spikes from synapses with PMS affinity that preceded the postsynaptic action potential are processed with a lower PMS concentration. In contrast, the spikes that arrive after the modulatory signal will be processed with a significantly higher plasticity factor. This would result in an asymmetric processing of the STDP learning window, leading to a continuous LTD in the affected synapses.

To satisfy the timing constraints the synaptic (axonal) delays are adjusted in such a way that the modulatory signal reaches the target neurons before the spike potentials (transmitted by the activated receptors). Figure 6.5 illustrates the required timings for the synapses between neuron PM, nociceptor P and motoneuron R.

Figure 6.5: Timing constraints for the subcircuit formed by motoneuron R, modulatory neuron PM and receptor P.

Figure 6.5 shows that the synaptic delay tP r from Nociceptor P to motoneuron R is larger than the synaptic delay tP pm from Nociceptor P to neuron PM added to the synaptic delay tP M rfrom PM to motoneuron R. This topology assumes that there is no delay inside neuron PM (during the activation and firing process). Otherwise the synaptic delay tP r would have to take neuronal delays into account.

The same timing constraints must be taken into account for the subcircuit involving the neuron PM, receptor F and motoneuron M. This is illustrated below in Figure 6.6.

Figure 6.6: Timing constraints for the subcircuit formed by motoneuron M, modulatory neuron PM and receptor F.

Figure 6.6 shows that the synaptic delay tF m from the reward-related sensor F to motoneuron M is larger than the synaptic delay tF pmfrom receptor F to neuron PM added to the synaptic delay tP M m from PM to motoneuron M. Again, as in Figure 6.5 this topology neglects any processing delay inside neuron PM.