UNA PERSPECTIVA INTERNACIONAL

The aim of this section is to investigate whether nodes can learn to cooperate using com- munications to achieve an objective. As such, an objective that requires communications to occur is devised and detailed in Section 3.5.1. The problem is tested in two Fleck3b motes and the findings discussed inSection 3.5.2.

3.5.1

The “PacketForwarder-RfmToLeds” Problem

This section details the design of an objective function that requires receiving of packets, transmission of packets and actuation. The design is detailed in Section 3.5.1.1 and its optimal solution provided inSection 3.5.1.2.

3.5.1.1 Packet Receiving-Sending Requirement

In the previous experiments, fitness was calculated and provided back to the evolution process entirely locally on the devices. However, since networked embedded have com- munications capability, they should, in theory, be able to learn to coordinate their be- haviour through communications to achieve an objective that is not local to the agent (i.e. fitness is either supplied from neighbouring devices or a system-wide objective function). To demonstrate this we desire a problem that requires devices to communicate, yet is well defined and preferably as simple as possible. Minimally, two devices (motes) are needed to demonstrate communications. For simplicity, these will be enumerated as Mote A and Mote B in this experiment.

RfmToLeds [1] is a simple radio program made popular by the TinyOS community as a tutorial and has often been used to demonstrate wireless communications to mote platforms. The program receives packets containing an incrementing counter, transmitted by another mote (typically IntToRfmM[1]), and simply displays the 3 least significant bits of the counter received using the 3 onboard LEDs.

This demonstrates receiving and actuating, however, since the packets generating the counter would need to come from an external source, it does not demonstrate transmis- sion of packets by the motes or communication between them. To require communication, a base mote (connected to a PC) executes the IntToRfmM logic, generating packets at a rate of 1 Hz. Importantly however, the packets are addressed only to Mote A (through specify a MAC destination address in the MAC layer). All combinations of LEDs are cy- cled through every 8 seconds. This is also used as the evaluation period for programs in the experiment.

any evolved logic) and used to calculate a fitness based on the counter value sent to Mote A and the LED status of Mote B. Frequent periodic fitness scores are supplied to both motes and summed locally over the execution time of 8 seconds per program. Importantly, communications is not explicitly rewarded, but rather communications must evolve in order to achieve the acceptable objective (LED state of Mote B given the counter sent to Mote A).

Ideally, the system would learn to receive the counter packet at Mote A, forward this to Mote B which in turn would receive it and set its LEDs accordingly, thus demonstrating the evolution of primitive communication. Therefore we term this as the “PacketForwarder- RfmToLeds” problem. Figure 3.13provides a diagrammatic representation.

The function set is limited to {radio_tx_int(x), set_leds(x), radio_rx_int()} since all of the essential aspects reside within this set. Based on these instructions, an optimal pair of programs (since the motes will require different behaviours) is shown inSection 3.5.1.2. The population structure was set to [5 3 10 2] 1] (5 elites, 3 highly ranked, 10 children, 2 randoms and 1 other). The mutation rate of set as 5% and single point crossover on code line (tree bases).

3.5.1.2 An Optimal Solution

Programs 7a and 7b were conceived as the minimal, ideal solution for the “PacketForwarder- RfmToLeds” objective.

Program 7 (a) and Program 7 (b) are the optimal programs for Mote A and Mote B respectively to the “PacketForwarder-RfmToLeds” objective.

radio_tx_int(radio_rx_int()) set_leds(radio_rx_int())

end end

Program 7a “PacketForwarder ” Program 7b “RfmToLeds”

Simply rewarding the devices for when the correct sequence is displayed on Mote B’s LEDs results in the local optima of Mote B leaving its LEDs unchanged for the entire evaluation period. This was ultimately negated by generating local negative fitness and supplying external (global) positive scores such that the inert behaviour (Mote B leaving

Base Node Fitness Objective Broadcast and receive programs

Wireless Sensor Node A Execution Thread Fitness O P ... Evaluation Thread while(1) Fitness-- Delay 100ms while(1) count++ FosMacSend(NodeA, count)

if NodeB.getleds() == (count & 7) NodeA.addScore(80) NodeB.addScore(80) Delay 1s Unicast MAC Msg − + Routing RPC Msg + Routing RPC Msgs radio_rx_int

Wireless Sensor Node B Execution Thread Fitness O P Evaluation Thread while(1) Fitness-- Delay 100ms −

radio_rx_int (Broadcast MAC) radio_tx_int (Broadcast MAC)

G H ... IDGP Framework Communications Evolved Logic Communications

Figure 3.13: Communications topology for the “PacketForwarder-RfmToLeds” experiment.

its LEDs unchanged for the entire evaluation), would yield an overall fitness of zero. This motivates the system to attempt to get more sequence matches during the evaluation period.

The final evaluation function can be expressed as follows:

• subtract 80 from the mote fitness for each evaluation (over 8 seconds)

• add 80 to the mote fitness for each correct LED sequence on Mote B matching the value sent from the base to Mote A

3.5.2

Results and Discussion

The novelty supplied by GP was underestimated for this experiment and served as a reminder of the importance of specifying the fitness function carefully. Initially, global negative fitness feedback was supplied to both motes when Mote B’s LED status did not match the acceptable pattern, and positive feedback supplied when it did match. While this appears reasonable, it did not consider that the motes’ communications use the same medium (radio frequency) as the framework that supplies the fitness feedback. Through the GP evolution, motes were able to evolve a “denial of service” strategy by congesting the channel, maximising packet collisions and ultimately reducing the number of received negative fitness packets. This converged to a fitness of zero, where no positive or nega- tive fitness was being received by the mote, however it was more optimal than the initial condition of receiving predominantly negative fitness. One can imagine this as the equiv- alent of a child learning that shouting when being reprimanded prevents the “negative feedback” from being heard.

Several approaches were experimented with to address this issue. Eventually it was determined that a combination of small, though frequent, negative feedback generated locally with positive larger global/external feedback, allowed the motes to converge to the optimal solution. The local frequent negative feedback is term “self deprecating” be- haviour. This combination does not “reward” motes that congest the channel since only positive fitness values are transmitted wirelessly.

With the updated fitness function employing both local negative and global positive feedback, the evolution proceeded more successfully. The evolutionary trajectory of the elite and pool fitness for both nodes is shown in Figure 3.14.

Again, the novelty of the generated solution was surprising. Both nodes were able to achieved elite fitness scores of 640 which is higher than the expected maximum fitness of 560 (8 × 80 − 80 × 1). This “cheating” appears to result from extra instructions during the evaluation prolonging the mote execution process just long enough so that the fitness accumulated occasionally receive 9 positive fitness values. Intriguingly, this strategy has a lower average (pool) fitness than that of Program 7(a) each time this strategy becomes

0 5 10 15 20 25 30 35 40 0 200 400 600 800 Generation Fitness Score

170 Elite Scores 170 Pool Average 187 Elite Scores 187 Pool Average

Figure 3.14: “PacketForwarder-RfmToLeds” fitness evolution

the elite and subsequently dominates the population. It was identified that this strategy relies on the evaluation period being near the beginning and end of the evaluation of the program. This is a risky strategy since most of the time it will not be beneficially aligned and the program may miss one or both of the positive feedback scores and will experience more local negative feedbacks. This causes the perpetual oscillation of both elite and pool fitnesses (lagging in phase) as the pool and elite toggle between the 2 solutions. Eventually the conservative strategy becomes elite again, however genetics of the risky strategy remain in the pool and eventually the conditions (timing) makes the risky strategy become the elite again perpetuates indefinitely explaining the oscillatory nature of the elite fitnesses displayed inFigure 3.14.

Nonetheless, the experiment did successful show the evolution of communications between nodes and also demonstrated the framework using a combination of both local and global fitness feedback simultaneously.