Using a bottom-up design, we develop a high-level abstraction of a low-level system. Control is applied on the top level with the idea that the low level will perform accordingly. Control of a hierarchical system can becentralisedor distributed. One may identify three control algorithms ranging from a centralised to a fully distributed system as shortly described below. • In centralised control, asupervisor controls the whole system by administrating the
entire physical distributed topology as well as the load status of each node.
• Infederated control, a node has supervisor rights over the nodes beneath it in the pro- cessing hierarchy.
• In distributed control, there is no supervisor and no control relationships between nodes. For instance, a failing node may broadcast an ‘S.O.S.’ message to the entire distributed control network and then listen for answers. Then each node evaluates the ‘S.O.S.’ message request and when it finds enough available resources will reply to the ‘S.O.S.’ owner. In the end, the failed node re-maps its failing task to the best fit node (in case of multiple ‘S.O.S.’ answers).
In this thesis we will only investigate centralised control (in the next chapter) because it is the simplest one to be implemented. One could say that if centralised control cannot be successfully implemented then distributed control will be even harder to be implemented.
6.3 Summary
In this chapter, we saw that distributed systems require, at a minimum, the following three functions: (1) mapping of the user applications onto a distributed environment (presented in Chapter 5), (2) distributed processing in the runtime (also presented in Chapter 5), and (3) re- sult retrieval from the distributed environment (already addressed in the field of distributed
systems, e.g., by Salton and Callan books [98, 99]). Moreover, when a distributed applica- tion works in a dynamic environment and assuming that the resources are limited, the system itself needs to (4) adapt to the environment changes in order to remain stable.
Chapter
7
A Simple Implementation of
Conductor Based on Centralised
Control
The previous chapter presented the CONDUCTORarchitecture. As a short summary, the CON- DUCTORarchitecture provides the following features: traffic processing, traffic splitting, and workload management. As a result, we build CONDUCTORon the following components that all interact: theprocessing node(Node) for traffic processing, theworkload manager(Work- Man) for resource accounting in every node, and theConductorfor system control by means of traffic routing and task re-mapping. Using these components, for the proof of concept of adaptive control, we built only the ‘centralised’ topology, which we will describe in this chapter. We also stress that this is a first attempt in which we deliberately simplified the very complex problem in order to be able to evaluate the main principle. It should not be seen as a full or mature solution.
Figure 7.1 depicts the ‘centralised’ case of CONDUCTOR. Given a distributed traffic pro- cessing system composed of a node hierarchy (N ode1.1,N ode2.1, etc.), we add a supervisor that monitors and controls the behaviour of the distributed traffic processing system with the help of workload managers (WorkMans) plugged in each processing node (W orkM an1.1, W orkM an2.1, etc.). The supervisor targets two global system goals (stability and perfor- mance) by the following types of control operations: (1) (re-)mapping of processing tasks onto the distributed traffic processing hierarchy and (2) (re-)routing of traffic streams. The first control type involves moving or replicating processing tasks from a congested node to a more lightly loaded node in order to prevent an effective node failure. The second control type comes as a need of the first: moving a task in different position in the processing hierar- chy often requires re-routing of the traffic feeding the task. Both control types work together in order to assureperformanceandstability.
Figure 7.1 shows an example of centralised control of a distributed processing architecture in a hierarchical topology.
T 0 T1 TN WorkMan T0 T1 TN WorkMan T0 T1 TN WorkMan T 0 T1 TN WorkMan T0 T1 TN WorkMan T 0 T1 TN WorkMan T1 TN T0 WorkMan Conductor
Input traffic
Node 1.1 Node 2.2Node 2.3
Node 3.2
Node 3.3 Node 3.1 Node 2.1
Figure 7.1: Centralised control of distributed processing architecture.
The distributed traffic processing system works as follows: the main stream (‘input traf- fic’) is first processed inN ode1.1 and primarily involves traffic splitting rather than traffic processing. Then, the split traffic (sub-streams of the main stream) flows in the next nodes of the processing hierarchy (N ode2.1, N ode2.2,N ode2.3). Some of these nodes are end- nodes (e.g.,N ode2.2) that perform traffic processing only and others are also splitters (e.g., N ode2.1, N ode2.3). When going down in the processing hierarchy we increasingly meet traffic processing nodes and fewer splitters. We assume that the end-nodes are properly cho- sen (in terms of hardware capacity for the incoming traffic stream) by the initial mapping according to the FPL program. For instance,N ode2.2is able to process the traffic split by the N ode1.1(one step splitting), whileN ode3.2andN ode3.3need two splitting steps.
In our example of distributed architecture illustrated in Figure 7.1 we can also see that we used two types of switches for traffic splitting: stand-alone switches such as the one located after N ode1.1, or built-in switches such as the ones inN ode2.1 andN ode3.1. Technically speaking, the former can be a standard Ethernet switch and the latter is a processing node with multiple output ports available. Supporting both switch types, stand-alone and built-in, provides flexibility in building a distributed traffic processing system by interconnecting of the best price/performance nodes.