Historically, a significant amount of work in distributed processing has been done in the literature to solve the so-called average consensus problem. Although the thesis addresses more general issues, we start with a description of this archetypal problem, as it allows to introduce the main ideas underlying many distributed algorithms.
We denote by T0,i ∈ R some scalar value observed by agent i ∈ V . The objective of average
consensus is, for any agent i, to estimate the average value over all ¯T0 = N1 PNi=1T0,i. One
of the most widespread approach to solve the problem distributively is the following. At time instant n, each agent i has an estimate Tn,i of the sought average. Each node i receives the
current iterates of other nodes in its neighborhood i.e., nodes j such that j ∼ i and updates its local iterate as a weighted average of its past iterate and the iterates received from neighbors. Formally, at each time n
∀i ∈ V, Tn,i=
X
j∈V
w(i, j)Tn−1,j (1.1)
where w(i, j) are non-negative weights such that w(i, j) = 0 whenever i and j are not con- nected. This condition ensures that the algorithm is indeed distributed among the graph. For simplicity, assume also that weights are non-negative and thatP
jw(i, j) = 1 for all i. Define
N × N matrix W whose coefficient (i, j) coincides with w(i, j). The above algorithm simply writes
Tn= W Tn−1
where Tnis the column vector whose i-th entry coincides with Tn,i. Note that W is a stochastic
matrix in the sense that all its rows sum to one. Distributed average consensus has its origins on applied statistics theory [48] and computer science [155],[156] (see [16] for a detailed state of the art on this subject). These conditions under which the above distributed algorithm converges to the sought average ¯T0are well known. Assume that
∀i ∈ V, X j:j∼i w(i, j) = 1 (1.2) ∀i ∈ V, X i:i∼j w(i, j) = 1 , (1.3)
that is, not only all rows of matrix W sum to one, but all columns also sum to one. Such a matrix W is called doubly stochastic. If W is moreover a primitive matrix1, it can be shown as an immediate application of the Perron-Frobenius theorem (see [81]) that
∀i ∈ V, lim
n→∞Tn,i= ¯T0. (1.4)
1
There exists m > 0 such that all coefficients of Wmare strictly positive. The assumption holds for instance if
Thus, the algorithm allows each node to eventually reach a consensus, an agreement on the final value. In addition, the value of the consensus coincides with the sought average ¯T0. The above
algorithm is sometimes referred to by some authors as a gossip algorithm.
The above gossip algorithm is synchronous in the sense that all agents must communicate their value at any moment of the algorithm, and the matrix W is fixed among the iterations. The Authors of [31] propose an asynchronous communication protocol. At time n, a given node chosen at random (say node i) wakes up and randomly select a node in its neighborhood (say node j). Nodes i and j merge their values by
Tn,i= Tn,j = 0.5Tn−1,i+ 0.5Tn−1,j, (1.5)
while other nodes k /∈ {i, j} keep their iterates Tn,k = Tn−1,k. The algorithm writes Tn =
WnTn−1where (Wn)nis a random sequence of matrices, namely Wn= IN− 0.5(ei− ej)(ei−
ej)T where IN is the N × N identity matrix, eiis the i-th vector of the canonical basis in RN
andT stands for transpose. The convergence (1.4) still holds (in the almost sure sense) under technical conditions specified in [31]. A key feature of the matrices Wn is that they are still
doubly stochastic for all n 6= 0.
The pairwise protocol described above can be considered as asynchronous in the sense that nodes are allowed to be inactive at some instants. However, it still requires some level of coor- dination among nodes: two nodes must simultaneously update their values at the same instant. Alleviating the need for such feedback in order to achieve truly asynchronous protocols has been an important stake in the years after [31]. The Authors of [10] propose a full asynchronous com- munication model called broadcast gossip. Similarly to the previous protocol, at each instant n one agent is randomly activated. The asynchronism is now at "agent-level" since the active agent i broadcasts its estimate to all neighbors without expecting a feedback transmission. Un- fortunately, it is shown in [10] that the sought convergence result (1.4) does not hold anymore. All that can be expected with such a simple protocol is a convergence in expectation, but not an almost sure convergence. We refer to [69] for more detailed considerations on that matter.
As a conclusion of this preliminary paragraph, the average consensus problem can be solved using a linear gossip algorithm in an asynchronous version, but the double stochasticity of the weighting matrices Wn is required at each n. Double stochasticity comes along with some
drawbacks regarding practical implementation, as it generally requires feedbacks in the network. Alternative works on that matter do not require double stochasticity at expenses of more complex models, e.g. [89], [84], [78]. For instance, [78] state the convergence result (1.4) by using only row-stochastic matrices, i.e. Wn1 = 1, but leading the set of communicating nodes grow at each
time n. It is worth noting the contribution of [89] that introduce the push-sum protocol (more generally analyzed by [34]). The gossip model of [89] allows to circumvent the convergence issue without the need of feedback links by introducing some additional communication, i.e. two variables instead of one are involved in the update step. [84] propose an asynchronous version of the latter model [89]. The convergence analysis and convergence speed are provided in [83] by the same Authors. We refer to [16] for a more complete description on general distributed average consensus algorithms.
1.3. Position of the thesis 35