ZAI 4. “Sierra de los Pinos” y ZAI 5. “Sierra Grande de Hornachos”

Total order protocols are used as a building block of some of the replication proto- cols describe earlier in Section 2.2. In fact the group based replication protocols of Section 2.2.3, using total order in the transaction’s dissemination are the simpler ones and also the ones requiring the smaller number of messages to be exchanged by the replicas.

In [DSU04] the authors studied total order protocols and proposed a classification of the total order protocols according to the mechanism used to order the mes- sages. They define five classes of ordering mechanisms: fixed sequencer, moving sequencer, privilege based, communication history and destinations agreement. Each of these classes represents a set of protocols that are likely to exhibit similar behavior.

In the fixed sequencer total order multicast protocols, the process sending a mes- sage sends it to all the destination processes and to the sequencer. After receiving the message the sequencer establishes the message order and multicasts the order to all destinations.

The moving sequencer total order multicast protocols are conceptually similar to the fixed sequencer ones. The difference is that the initial message is multicasted to all sequencers instead of to the fixed sequencer. When receiving the message, only one of the sequencers orders it and multicasts the order to all of its destina- tions, informing all the other sequencers of the message it ordered.

Since these classes of protocols are conceptually similar, we merged them into a single class named of sequencer based total order protocols.

The privilege based total order multicast protocols assume the existence of a token. The token circulates among all processes and grants to its owner the per- mission to multicast and order messages. Messages multicasted while the process is not the token holder are queued. When receiving the token, the process orders all messages in its queue and multicasts them. Afterwards it sends the token along with the current ordering version to the next token holder.

Regarding the communication history based total order multicast protocols, in the ones using causal history, each process has a timestamp which it increases by 1 when sending a message, attaching it to the message being sent. In the reception of a message, the process timestamp is increased by 1 if the process timestamp is higher than the timestamp of the received message. If the timestamp of the received message is higher than the local timestamp, then the message timestamp is increased by 1 and becomes the process timestamp. Messages are delivered according to their timestamp and messages with the same timestamp are ordered

2.3. TOTAL ORDER PROTOCOLS 23 according to to the identifier of its sender. These protocols can only deliver a message m after receiving from every process, a message that was multicasted after the reception of m, or that is concurrent with m, i.e., that has the same timestamp as m.

The destinations agreement total order multicast protocols require that all pro- cesses agree on the messages order. Their agreement can be on the message order, on a message set or on the proposed message order. In the protocols using agree- ment on the message order, the sender multicasts the message to be ordered to all processes, which will assign a local timestamp to it and multicast this timestamp to all processes. When the local timestamp is received from all processes the mes- sage global timestamp is established and messages are delivered according to the global timestamp.

From the algorithms description, the communication patterns, as well as the ex- pected execution latency of each protocol class can be established. The commu- nication patterns and expected latencies of the protocols execution are presented in Table 2.1, where n and m are number of processes, with m < n, k is the number of messages queued and l is the point to point latency.

Sequencer Privilege Communication Destinations

based history agreement

Comm. n + n 1 + (n_k|n) n + m × n n + n × n Pattern Latencya l + l n₂l l l + l Latencyb _{2l +}6l n n+6 2 l 4l 4l + 4l

a _{Assumes that point to point latency between any two processes is l.}

b _{Assumes that point to point latency between n − 1 of the processes is l, and that}

the remaining one has a point to point latency of 4l.

Table 2.1: Total order protocols communication pattern and latency.

In terms of communication pattern, the communication history is the protocol requiring fewer messages when all processes send messages at the same rate, re- quiring a single multicast message to order a message (i.e., m = 0). It may require some additional messages (i.e., m 6= 0), when some process, m, send messages less frequently than the others, in order to avoid latency increases.

The privilege based protocol requires a point to point message from the token holder to the next token holder and i) a multicast message from the token holder to multicast and order all the messages in the queue; or ii) k multicast messages to multicast and order each of the k messages in its queue. When several messages

are multicasted and ordered in a single multicast message, the privilege based is the protocol requiring fewer messages to order a set of messages.

The sequencer based protocol requires two multicast messages to order a mes- sage, i.e., a multicast message from the sender to all destinations and a multicast message from the sequencer to all destinations.

The destinations agreement protocol requires a multicast message from the sender to all destinations followed by a multicast message from each destination to all the destinations to send the ordering info.

Regarding the protocols latency, in a scenario where the processes point to point latencies are equal, the communication history is the one with the lowest latency, in a optimal scenario where all processes are sending the same number of mes- sages and at the same message rate. In this scenario, the protocol latency equals the point-to-point latency.

The sequencer based and destinations agreement protocols present similar laten- cies which are twice as large as the point to point latency. In the sequencer based it is the latency of a message from the sender to the sequencer plus the latency from the sequencer to all destinations. In the destinations agreement the latency is the sum of the latency of the message from the sender to all destinations plus the latency of the message sent from each of the destinations.

In the privilege based protocol, the latency of the protocol depends on the instant a message is sent. if it is sent immediately after the token is passed away then the latency will be equal to the time it takes to the token to reach this sender again. If the message is sent immediately before being the token holder, then the latency will be close to 0. Considering the middle case, i.e. a scenario in which messages are sent at a constant rate, then the latency will be equal to (n)₂ l.

Considering a scenario in which a process point to point latency is four times higher than the other processes point to point latency, this single process with higher latency will affect the latency of all protocols. The sequencer based pro- tocols are the ones which the smallest increase in the protocol latency. In the privilege based protocols the token period will be increased by the latency of the process with higher point to point latency, and the communication history and destinations agreement protocols will observe a latency increment proportional to the highest point to point latency among every two processes.