.
Student: Juli´an Barreiro-G´omez. Advisor: Nicanor Quijano, PhD.
Contents
1 Introduction 1
2 Distributed Optimization Based on Population Dynamics and Graphs. 2
2.1 Preliminaries . . . 2
2.2 Framework . . . 3
2.3 Population Dynamics . . . 4
2.4 Optimization problems with constraints . . . 6
3 Consensus based on Population Dynamics. 11 3.1 Consensus . . . 11
3.2 Tracking Consensus . . . 13
3.2.1 Tracking Consensus with delay . . . 13
3.3 Tracking Output Consensus . . . 14
4 Distributed Control Based on Population Dynamics for Drinking Water Networks: Ap-plication Case Study. 15 4.1 System management criteria . . . 15
4.2 DWN Model and constraints . . . 16
4.3 Optimization problem . . . 16
4.3.1 Partition of the network . . . 17
4.4 Results . . . 17
5 Conclusions 18
1
Introduction
Distributed decision-making systems have gotten special importance in the design of large scale processes. Large scale systems are commonly compound by many subsystems affecting their behavior each other by coupling conditions, this coupling makes appropiate to consider the total system behavior as a whole instead of have control systems for each sub-system independently. The main motivation in developing distributed strategies applied to these kind of systems, is the lack of centralized information, the cost of having access to full information, computational time to process large amount of data and more implications associated to it like communication faults. In addition to this, management of large amount of information would imply high computational costs becoming a relevant issue for real-time control design.
There are many applications with characteristics above mentioned and which could be represented as a network, what incentives to focus on new and better decision making strategies and that could consider a total behavior (in the sense that all states throughout the network affects a global objective) based on local available information [1, 2]. Two of the most relevant approches considered for distributed design systems, are distributed optimization and distributed agreement or consensus [3]. The former approach in applied to engineering application that might demand to find a value in which subsystems must agree [4] whereas the latter is applied to other cases whose main objective is to maximize or minimize a cost function.
Evolutionary game theory [5] has been a significant theory in the solution of engineering problem as multi-agent learning [6] and optimization problems considering constraints and limitations in information given by a graph [7][1]. This paper proposes some dynamics in order to solve optimization problems with
constraints and consensus in a distributed way based on local information established by a graph represent-ing a network and makes analysis on stability usrepresent-ing Lyapunov theory [8].
First, preliminaries about graph theory and evolutionary dynamics provides concepts that are used further, then proposed dynamics are presented and stability analysis shows convergence to equilibrium point. At second part, different optimization problem forms are presented and some examples are solved to illustrate population dynamics approach. Afterwards; consensus, tracking consensus (with and without delay) and output tracking consensus with population dynamics are solved and compared with another algorithm. Finally, a large-scale engineering application is designed based on all ideas proposed in this document.
2
Distributed Optimization Based on Population Dynamics and
Graphs.
2.1
Preliminaries
LetG= (V,E) be an undirected connected graph, whereV is the set of vertices andE={(i, j) :i, j∈ V}is the set of edges. The graphGexhibits the topology of a society where the setV represents theN strategies of a gameS ={1, ..., N}and the setE determines possible interactions among strategies in the society. Society consists of M populations P = {1, ..., M} one for each clique of G. Let C ={Cp : p∈ P} be the
set of cliques of G representing all populations in society where Cp = (Vp,Ep) andVp ⊂ V represents the
Np strategiesSp={i:i∈ Vp} of populationp∈ P. Aditionally, all cliques must complete the total graph
∪p∈PCp=G.
In this paper, we asumme that cliques are already known. There are some methods to solve the maximum clique problem, for instance the Bron Kerbosch algorithm [9] or other alternatives like based on replicator dynamics [10].
For a node i∈ V, we define the amount of cliques that contain the nodei, denotedG(i). G(i) =X
p∈P
g(i, p) (1)
g(i, p) =
1 ifi∈ Vp
0 otherwise (2)
Since G is a connected graph, all cliques must share at least one node each other denoted as intersection nodes. Ip = {i ∈ Vp : G(i) > 1} is the set of nodes which are intersection in a population p ∈ P and
I=∪p∈PIp is the set of intersection nodes inG.
The set of social states is given by a simplex denoted ∆ which is a constant set and the set of states of populationp∈ P is given by a non-constant simplex denoted ∆p.
∆ ={x∈RN+ :
X
i∈S
xi=m} (3)
∆p={xp∈RN
p
+ :
X
i∈Sp
xpi =mp} (4)
Where mis the constant mass of agents in the society and mp the mass of the populationpwhich varies
over time with associated dynamics. There is a relation between social and population states considering thatxpi = 0 ifi /∈ Vp.
xi=
1 G(i)
X
p∈P
xpi (5)
LetFi : ∆→Rbe the payoff function for the proportion of agents playing strategyi∈S andFip: ∆p→R
the payoff function for the proportion of agents playing strategy i∈Sp. The fitness for a strategyi∈S is
the same as the fitness for a strategyj∈Sp ifi=j, consequentelyF
Society average function denoted ¯F(x) and the average function for all populations p∈ P denoted ¯Fp(xp) are given by.
¯
F(x) = 1 m
X
i∈S
xiFi (6)
¯
Fp(xp) = 1 mp
X
i∈Sp
xpiFip (7)
As there is a relation between social and population states, there is a relation between masses to satisfy ∆. This relation is shown in Lemma 1, equation (8).
Lemma 1: m6=P
p∈Pm
psince intersections nodes affect this equality. The relation among social mass
and population masses is given by
m=X
p∈P
mp−X
i∈S
(G(i)−1)xi (8)
Proof: Replacingmp and then applying the fact thatxpi = 0 ifi /∈ Vp.
m=X
p∈P
X
i∈Sp
xpi −X
i∈S
(G(i)−1)xi
=X
i∈S
X
p∈P
xpi −X
i∈S
(G(i)−1)xi
=X
i∈S
G(i)xi−
X
i∈S
(G(i)−1)xi
=X
i∈S
xi
2.2
Framework
This section presents the framework of this paper. The main assumptions are: i)F(x) is full potential game. ii)F(x) is a stable game.
Definition 1: F(x) is a full potential game if there is a continuously differentiable functionf(x) known as potential function satisfying
∂f(x) ∂xi
=Fi(x) for alli∈S, x∈∆ (9)
For each populationFip(xp) =Fi(x) ifxi=xpi for alli∈S
p and restrictions about information dependency
areFip({information aboutj}:j∈ Cp) ifi /∈ I andFp
i(information abouti) ifi∈ I.
Definition 2: Tangent space for strategic interaction in society is defined asT∆ ={z∈RN :Pi∈Szi=
0}. IfF(x) is continuously differentiable, F(x) is a stable game if the Jacobian matrixDF(x) is negative semidefinite with respect toT∆∀x∈∆ [11]
z>DF(x)z≤0 for allz∈T∆ andx∈∆ (10) Moreover the characteristic of a potential function can imply satisfying conditions for a stable game.
Lemma 2: If the potential functionf(x)∈C2 of the full potential gameF(x) is concave, thenF(x) is
a stable game.
Proof: f(x)∈C2 andFi(x) = ∂f∂x(x)
i . Then, iff(x) is concave, the Hessian matrixH =DF(x) is negative semidefite andz>DF(x)z≤0 as required.
Another important fact related to stable games, is the behavior when varying mass once game is on a Nash equilibrium.
Lemma 3: Let F(x) be a stable game and x∗ ∈ ∆ be the Nash equilibrium where ∆ = {x ∈ RN :
P
i∈Sxi=m}. If massm increases (decreases) to a greater (smaller) mass ˜m, thenxi increases (decreases)
to a greater (smaller) state ˜xi for alli∈S.
Proof: Ifx∗∈∆ thenFi(x) =Fj(x) = ¯F(x) for alli, j∈S. Then if massmincreases (decreases) all states
xi increase (decrease) to converge to a new value Fi(x) = Fj(x) = ¯F(x) for all i, j ∈S since all fitnesses
have the same tendency as in a stable game where fitness functions are decreasing.
2.3
Population Dynamics
There is a graphGdescribing a network in which there is not complete information. The network is divided intoMcliques which are sub-complete-graphsCpwithp∈ Pwhere full information is guaranted by definition.
The objective for society is converging to a Nash equilibrium in the society denoted x∗ ∈ ∆. At each populationp∈ Prepresented by a clique, there is convergence to a Nash equilibrium denotedxp∗∈∆psuch
that P
i∈Spx
p
i =mp where the population massmp varies over time, then it is necessary thatxp→∆p as
t→ ∞.
A game is solved for each population with constraints given by massesmp which will vary dynamically.
There areM dynamics of this type, one for each cliqueCp for allp∈P.
˙ xpi =xpi
F
p i −F¯
p−β
1 mp
X
j∈Sp xpj−1
,∀i∈Sp (11)
Whereβ is a factor determining convergence to simplex ∆p when xp ∈/ ∆p and trajectories are faster as β is increased. On the other hand, there are dynamics for population massesmpdecribing agents’ movements among populations through cliques’ intersections. There are dynamics of this type for all nodes whosei∈ I and it is considered thatmqi = 0 ifq /∈ Vq.
˙
mpi =mpi
xi−x p
i −β
1
κi+ (G(i)−1)xi
X
q∈P
mqi |Iq| −1
,∀p:i∈ V
p (12)
Where β determines a rate of convergence to conditions on population masses to fulfill ∆ given by κi+
(G(i)−1)xi andκiis a distribution ofm, thenPi∈Iκi=m. There is a relation betweenm p
i andm
p. Then
mp varies dynamically sincemp
i has dynamics.
mp= 1 |Ip|
X
i∈Ip
mpi (13)
For these dynamics vectors of masses are defined for alli∈ I asmi={mipfor allp:i∈ Vp}and vectors of
population states for alli∈ I asxi={xpi for allp:i∈ Vp}. Note thatmi6=mi andxi6=xi.
It is neccesary to show that a total problem represented by the graphG as a society is solved through this division into cliques as populations where masses of agents migrate from one population to other.
Proposition 1: Ifxp∗in (11) for allp∈ P andm∗i in (12) for alli∈ I, then equivalent social states are x∗∈∆.
Proof: The first premise implies that for anyp∈ P,Fip(xp) =Fp j(x
p) for alli, j∈Sp andP
i∈Spx
p
i =m
p.
for allr, k∈ P consequentelyx∗= (1/G(i))P
p∈Px
p i
∗
. Aditionally,m∗i for alli∈ I in equation (12) implies that
κi+ (G(i)−1)xi=
X
p∈P
mpi
|Ip|, ∀i∈ I, p:i∈ V p
X
i∈I
(κi+ (G(i)−1)xi) =
X
i∈I
X
p∈P
mpi |Ip|
X
i∈I
κi+
X
i∈I
(G(i)−1)xi=
X
p∈P
mp
(G(i)−1) = 0 for alli /∈ I, then a sum can be replaced m+X
i∈S
(G(i)−1)xi=
X
p∈P
mp
According to Lemma 1, thenx∗∈∆ completing proof.
Since masses of agents move from one population to other, then the feasible region at each clique varies ∆p for allp∈ P, and it must be shown that trajectories converge to equilibrium points and that states track corresponding simplex.
Theorem 1: For dynamics given by equations (11) and (12) and F(x) satisfying condition (10) to be a stable game,xp∗for allp∈ Pandm
i∗for alli∈ I are globally asymptotically stable with region of atraction
in the positive orthant.
Proof: Using the convex radially unbounded Lyapunov function V(xp,m
i) whose minimum is zero and
coincides on (xp∗,m
i∗)
V(xp,mi) =
X
i∈Sp
xpi −xpi∗
1 + ln
xp
i
xpi∗
+
X
p:i∈Vp
mpi −mpi∗
1 + ln
mp
i
mpi∗
• V(xp∗,m
i∗) = 0
• V(xp,mi)>0 ifxp6=xp∗ ormi6=mi∗
Then, derive is
˙
V(xp,mi) = X
i∈Sp
1−x
p i ∗
xpi
˙
xpi + X
p:i∈Vp
1−m
p i ∗
mpi
˙ mpi
For analisys, all|Iq| ≥1∀q∈ Pand for simplicity making the variable change ˜κ= min p:i∈Vp(κ
p
i) + (G(i)−1)xi
for masses dynamics since those term are always positive
˙
V(xp,mi)≤xp>−
1 mpx
p>1xp>F(xp)
− β mpx
p>1xp>1+xp>
1β−
xp∗>+ 1 mpx
p∗>1xp>F(xp) + β
mpx
p∗>1xp>1−xp∗>1β+
m>i 1x>i 1 1 G(i)−m
>
i xi−m>i 1m
>
i 1
β ˜ κ+m
>
i 1β−m
∗
i
>
1x>i 1 1 G(i)+
m∗i>xi+m∗i
>1m>
i 1
β ˜ κ−m
∗
i
Letxp>1=mp+ where could be positive or negative depending on whether xp is geometrically above or below the simplex ∆p, xp>1 > 0 and 1is a column vector of ones, whose cardinality is |1| =Np for population states and|1|=|Ip| for intersection nodes with masses dynamics. Alsom>
i 1>0 since masses
are positive and not necessarily these masses satisfy condition respect to equation (8), then this can me consider asm>i 1= ˜κ+γwhere γcould be positive or negative. Finally we obtain that
˙
V(xp,mi)≤
1−m
p+
mp
xp>F(xp)−β
2
mp + (x p−xp∗
)>F(xp) + γ
G(i)x
>
i 1+ (m∗i −mi)>xi−β
γ2
˜ κ
i) if= 0 andγ= 0⇒ ˙
V (xp,m
i) ≤ (xp−xp∗)>F(xp) + (m∗i −mi)>xi. The first term is negative since F(xp) is a stable game
according to equation (10) and the second term is also negative according to Lemma 3 and figure 1. Then ˙
V (xp,m i)≤0.
ii)if6= 0 andγ6= 0⇒there is aβ such that ˙V (xp,m
i)≤0 given by
β≥ m
p˜κ
˜ κ2+mpγ2
1−m
p+
mp
xp>F(xp) + (xp−xp∗)>F(xp) +
γ G(i)x
>
i 1+ (m∗i −mi)>xi
Equality holds whenxp =xp∗ andm
i=m∗i, applying LaSalle’s Invariance Principle every solution starting
inRNp
+ andR
G(i)
+ approaches toxp∗ andm∗i ast→ ∞.
m2i
x∗
i={x1i∗, x2i∗}
x∗
i={x1i∗, x2i∗}
m1i mi
m∗
i
Figure 1: Geometric notion for masses dynamics and intersection states.
2.4
Optimization problems with constraints
One of the main characteristics of full potential games is that Nash equilibrium points correspond to extrema of the potential function for which these points satisfy Karush-Kuhn-Tucker first order condition. Moreover, if potential function is concave then games are stable and an optimization problem can be solved in a distributed way with population dynamics and masses dynamics shown in Section 2.3. Some optimization problem forms are set and illustrative examples are solved.
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 0 100 200 300 400 500 600
Time [seg]
xi wi th 1 ≤ i ≤ 13 an d
13X i=1
xi
=
52
2
Evolution of society states
Society´s states Sum of all society´s states
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
−80 −60 −40 −20 0 20 40 60
Time [seg] Fi ( x ) w it h 1 ≤ i ≤ 13 Fitness functions Non−intersection nodes Intersection nodes a) b)
0 2 4 6 8 10 12 14 16 18 20
0 50 100 150 200 250 300 350 400 450
500 Evolution of society states
Time [seg]
St at es x an d
10X i=1
xi
0 2 4 6 8 10 12 14 16 18 20
−400 −200 0 200 400 600 Fitness functions
Time [seg]
Fi ( x ) w it h 1 ≤ i ≤ 10 c) d)
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
0 10 20 30 40 50 60 70
Evolution of society states
Time [seg]
xi wi th 1 ≤ i ≤ 13 , x1 + x2 = 50 an d x4 + x5 = 40
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
−30 −20 −10 0 10 20 30 40 50 60
Fitness functions (Lagrangian)
Time [seg]
∇x L ( x , µ ) an d ∇µ L ( x , µ ) e) f)
Figure 2: Optimization problems results.
shown. Consider this non-linear optimization problem: maximize f(x) subject to
N
X
i=1
xi=m
xi≥0 where 1≤i≤N (14)
Where x∈ RN+, f : RN+ → R and m ∈R+. We assume that f is continuously differentiable f ∈C1 and
concave. For this optimization problem, there is a full potential game given byF(x) =∇f(x) and due to the fact thatf(x) is concave thenF(x) is a stable game according to Lemma 2.
First constraint in this optimization problem determines the set of social states ∆ and the second con-straint is satisfied since states arex∈RN+. Constraints about information affect dependency among
Illustrative Example
Consider the quadratic optimization problem given by:
maximizef(x) =−x2
1−x
2
2−x
2
3−x
2
4−x
2
5−x
2
6−x
2
7−x
2
8−x
2
9−x
2
10−x
2 11
−x212−x213+x1x2+x6x7+x9x10+x11x12+x12x13+ 50x1
+ 40x2+ 12x3+ 30x4+ 60x5+ 70x6+ 85x7+ 50x8+ 21x9
+ 35x10+ 26x11+ 29x12+ 32x13
subject to
13
X
i=1
xi = 522
xi≥0 where 1≤i≤13 (15)
The society topology is shown in figure 3. Then, it can be verified that condition in Definition 1 is satisfied for this optimization problem given the graphG.
1
2
13
3
12
4 5
11 6
7
8 9
10
Figure 3: Society graphG with 4 cliques and 2 intersection nodes.
Figure 2a. shows the evolution of social states xi for alli ∈ S converging to solution x∗ ∈Delta and
social massmsatisfying constraint for optimization problem. Figure 2b. showsF(x) =∇f(x) for alli∈S converging to average fitness ¯F.
On the other hand, a less restrictive optimization problem can be set. This optimization problem only demands the positiveness of variables and from a game theoretical perspective implies a variation of social mass. Consider this non-linear optimization problem:
maximize f(x)
subject to xi≥0 where 1≤i≤N (16)
Wherex∈RN+, f :R+N →R. We assume thatf is continuously differentiable f ∈C1, concave and whose
maximum is located in the positive orthant.
For this optimization problem, there is not a constraint related to the social mass. Then, equivalence shown in Lemma 1 is not longer required and therefore part of masses dynamics that guarantees satisfaying a social mass constraint presented in Proposition 1 is neither required. Accordingly with the mentioned, equation (12) is modified for alli∈ I
˙
mpi =mpi (xi−x p
i),∀p:i∈ V
p (17)
and equilibrium point m∗i implies that xi =x p
i for all p∈ P considering that x p
i = 0 for alli /∈ V p. This
system continues converging to a Nash equilibriumx∗ with an arbitrary social massm∈RN
+. On the other
hand, solution for optimization problem (16) is given by a social statexsuch thatF(x) =∇f(x) = 0 since it is known that solution is an interior point. There is a known ˜xi so that Fi(˜xi) = 0 for an i ∈ I, then
In order to guarantee that convergence value for fitnessesF(x) = 0, it is guaranteed thatxpi →x˜i for all
p:i∈ Vp by modifying relation in (5) only for ani∈ I considering thatxp
i = 0 for alli /∈ V p
xi=
1 G(i) + 1
X
p∈P
xpi + ˜xi (18)
Remark 1: In case that ˜xi is not easily found for anyi ∈ I, it is possible to establish any decreasing
ficticious function ˜F(xN+1) where xN+1 is an intersection node, and ˜xN+1 is known so that ˜F(˜xN+1) = 0.
This addition of a new variable does not affect the solution, but allows to force trajectories to converge to desire value zero.
Theorem 2: For dynamics given by equations (11) and (17) and F(x) satisfying condition (10) to be a stable game,xp∗for allp∈ Pandmi∗for alli∈ I are globally asymptotically stable with region of atraction
in the positive orthant.
Proof: Using the same Lyapunov function from theorem 1. ˙
V(xp,mi) =xp>−
1 mpx
p>1xp>F(xp)
− β mpx
p>1xp>1+xp>
1β−
xp∗>+ 1 mpx
p∗>1xp>F(xp) + β
mpx
p∗>1xp>1−xp∗>1β+
m>i 1x>i 1 1 G(i)−m
>
i xi−m∗i
>
1x>i 1 1 G(i)+m
∗
i
>
xi (19)
For this analysis we have1x>i 1G1(i) = ˜xi, and1is a column of ones with appropriate dimension for population
states and masses. ˙
V(xp,mi) =
1−m
p+
mp
xp>F(xp)−β
2
mp + (x p
−xp∗)>F(xp) +
(m∗i −mi)>(xi−˜xi) (20)
˙
V (xp,mi)≤0 when= 0, and there is aβ when6= 0 given by
β ≥
mp 2
1−m
p+
mp
xp>F(xp)−β
2
mp + (x p
−xp∗)>F(xp) +
(m∗i −mi)>(xi−˜xi)
(21)
Illustrative Example
Consider the following non-linear optimization problem.
maximizef(x) =−x2
1−x
2
2−x
2
3−x
2
4−x
2
5−x
2
6−x
2
7−x
2
8−x
2
9−x
2
10+
500x1+ 400x2+ 120x3+ 300x4+ 600x5+ 700x6+
850x7+ 200x8+ 210x9+ 230x10
subject to xi≥0 where 1≤i≤10 (22)
Constraints about information sharing are represented by the graph G in figure 4, with the posibility to impose a value in the intersection node 3.
1
2 3
r
4 5 6 7
8 9
10
Figure 4: Society graph G with 3 cliques and 2 intersection nodes. Imposed value r at node 3 shows the known value ˜xi as a reference
Figure 2c. shows evolution of social states with two different initial conditions, it can seen that trajectories converge to the solution. Fitness functionsF(x) =∇f(x) converges to zero from different initial conditions (Figure 2d.)
Suppose that there is an strategic interaction with more than one constraint, for instance, it is wanted to converge to an equilibrium point such that the total amount of certain groups of agent proportions are constant. Consider this non-linear optimization problem:
maximize f(x) subject to Ax=b
xi≥0 where 1≤i≤N (23)
Where x ∈ RN+, f : R
N
+ → R. We assume that f is continuously differentiable f ∈ C
1 and concave.
A∈RO×N andb∈RO
For this optimization problem, µis the Lagrange multiplier vector and the Lagrange function l :RN × RO→Ris
l(x, µ) =f(x) +µ>(Ax−b) (24) ∇xl(x, µ) =∇f(x) +A>µ (25)
∇µl(x, µ) =−Ax+b (26)
The Lagrange condition is used to find possible extremizers [12] in which
∇xl(x, µ) = 0 (27)
∇µl(x, µ) = 0 (28)
Consequentely, fitness functions for each node are chosen asF(x) =∇xl(x, µ) andF(µ) =∇µl(x, µ) and it is
solved by using references as in the previous optimization problem in intersection nodes for social states and for Lagrange multipliers. For both, F(x) andF(µ) a ficticious function can be set as explained in Remark 1.
Lemma 4: If the function is continuously differentiablef(x)∈C2 and concave and constraints are of
the formAx=b, then the gamesF(x) =∇xl(x, µ) andF(µ) =∇µl(x, µ) are stable approving theorem 2.
Proof: If the concavef(x) is continuously differentiablef(x)∈C2and constraints are of the formAx=b,
Lagrangian function is
l(x, µ) =f(x) +µ>(Ax−b) Then, fitness functions are given by
F(x) =∇f(x) +A>µ F(µ) =−Ax+b
DxF(x) = ∇2f(x) that is the Hessian matrix H ≤0 since f(x) is concave, and DµF(µ) = 0. Therefore
F(x) andF(µ) are stable games according to Definition 2.
Illustrative Example
Consider the following non-linear optimization problem.
maximize f(x) = (25−x1)2+ (20−x2)2+ (15−x3)2+ (10−x4)2
+ (5−x5)2
subject to x1+x2= 50
x4+x5= 40
xi≥0 where 1≤i≤5 (29)
Figure 5 shows social topology with 2 populations and Lagrange multipliers nodes for constraints. For this optimization problem a ficticious function is set in an intersection node between nodesµ1,µ2thenF(µ)
is forced to be zero.
1
µ1
2
3
5 4
µ2
r
Figure 5: Society graphGwith 2 cliques and an intersection node, and dependency information for Lagrange multipliersµ1,2. Imposed valuershows the known value ˜xi as a reference
Figure 2e. shows evolution of social states and the sum of states for each constraint. It can be seen that there is a transitory even in which constraint are not satisfied, then trajectories of social states get out the feasible region and converge back to this set. Figure 2f. shows fitness functions, solid lines correspond to∇xl(x, µ) and dotted lines correspond to∇µl(x, µ) converging to zero satisfying KKT first order condition.
Finally, other optimization problems can be considered and solved in the same manner. For instance, consider the optimization problem given by
maximize f(x) subject to Ax=b
Cx≤d
xi≥0 where 1≤i≤N (30)
Where x ∈ RN
+, f : RN+ → R. We assume that f is continuously differentiable f ∈ C1 and concave.
A ∈ RO×N, b ∈ RO, C ∈ RQ×N and d ∈ RQ. Then, adding a vector of slack variables s ∈RQ+ to states
vector ¯x= (x>|s>)>, a new problem of the form of problem (23) is obtained. In general, more optimization
problems as in [13] can be solved with these population dynamics when forcing a zero for gradients solution is guaranteed.
3
Consensus based on Population Dynamics.
3.1
Consensus
In this section, consensus using population dynamics proposed in Section 2.3 is presented and results are compared with other consensus algorithms proposed in [14] [15]. Consensus is generally applied for
obtain-ing all states in a system with a same value, while in population dynamics convergence is given by a Nash equilibrium, that in case it is a mixed strategy equilibrium implies that all fitnesses get the same value. Furthermore, it is possible to obtain a Nash equilibrium with all states getting the same value as long as all fitnesses are equal.
An algorithm for consensus depends on graph topology and as required in population dynamics proposed in this paper, graph should be connected. An algorithm to achieve consensus is
˙ xi =
N
X
j=1
aij(xj−xi) (31)
whereaij is an element in the adjacency matrix ofG andN is the total amount of nodes|V|.
1 2 3 4 5 6
7
8
9
10
11
12 13
14 15
16
GraphG
1 2 3 4 5 6 7 8 9 10 11
12
13
14
15
16
17
18
19
20
21 22
23 24 25
26 27
28 29
30
GraphG
a) b)
Figure 6: Graphs for consensus problem. Node 16 as reference.
Consider the graphGin figure 6a. with 16 nodes, 31 links and 6 cliques. It is wanted to obtain a consensus of states without any reference or restriction. In order to make consensus with population dynamics equations (11) and (17) without ficticious function are implemented, fitnesses are considered to beFi(x) =−xi then it
is guaranteed thatF(x) is stable and that solution is achived whenxi=xj for alli, j∈S. The convergence
value depends on initial conditions for social states and masses states. Results are shown in figure 7
0 0.5 1 1.5 2 2.5 3 3.5 4
0 2 4 6 8 10 12 14 16 18
Consensus with population dynamics
Time [seg]
St
at
es
xi
0 0.5 1 1.5 2 2.5 3 3.5 4
0 5 10 15
Consensus with adjacency matrix
Time [seg]
St
at
es
xi
a) b)
Figure 7: a) Consensus using population dynamics without a fixed social mass. b) consensus with adjacency matrix
3.2
Tracking Consensus
Tracking consensus is a consensus in which all states converge to the same desired value and it is achived by fixing a state without dynamics, then equation (31) is written as
˙ xi=
N
X
j=1
aij(xj−xi)−ai(N+1)(xj−x)˜ (32)
where ai(N+1) = 1 at the adjacency matrix indicates that an agent has information from the reference
agent ˜x.
The graphGshown in figure 6b. has 30 nodes with 8 cliques. The objective is to make consensus tracking state of node 16 which has only one link. For population dynamics approach, node 16 works as the reference for the intersection node 8 where its value is imposed through masses dynamics as explained in optimization problem (23) making fitnesses converge to zero. In this case, as fitnesses areFi(x) =−xi the value of ˜xset
the value for the whole network.
Desired value for tracking consensus is ˜x= 20. Figure 8a. shows results with population dynamics and 8b. shows results with tracking consensus using adjacency matrix.
0 0.5 1 1.5 2 2.5 3
0 5 10 15 20 25 30 35
Consensus with population dynamics
Time [seg]
St
at
es
xi
0 0.5 1 1.5 2 2.5 3
0 5 10 15 20 25 30
Consensus with adjacency matrix
Time [seg]
St
at
es
xi
a) b)
Figure 8: a) Tracking consensus using population dynamics and ˜x = 20. b) Tracking consensus with adjacency matrix and virtual agent ˜x= 20
3.2.1 Tracking Consensus with delay
Consensus and tracking consensus presented above assume that needed information is available without time delay. Now consider that when a particular agent requires information from another agent, this information has a delayτ. In the same way, when a proportion of agents playing in a clique require information about other specific strategies to get an average fitness, this information has a delayτ. Then, consensus based on adjacency matrix and population dynamics are written as
˙ xi(t) =
N
X
j=1
aij(xj(t−τ)−xi(t))−ai(N+1)(xj(t−τ)−x)˜ (33)
˙
xpi(t) =xpi(t)
F
p i(t)−F¯
p(t−τ)−β
1 mp
X
j∈Sp
xpj(t−τ)−1
,∀i∈S
p (34)
As an example, for graph 6b. assume that there is delay only for one agent or strategy. Then this delay isτ= 5secfor information related toi= 1,i∈ V. Figure 9 shows that consensus with population dynamics is robust against delay whereas consensus based on adjacency matrix convergence is slow owing delay and trajectories change everyτ.
0 1 2 3 4 5 6 7 0
5 10 15 20 25 30 35
Consensus with population dynamics with delay
Time [seg]
St
at
es
xi
0 5 10 15 20 25 30 35 40 45 50
0 5 10 15 20 25 30
Consensus with adjacency matrix with delay
Time [seg]
St
at
es
xi
a) b)
Figure 9: a) Tracking consensus using population dynamics, ˜x= 20 and delay τ= 5secfori= 1, i∈ V. b) Tracking consensus with adjacency matrix, virtual agent ˜x= 20 and delayτ = 5secfori= 1, i∈ V.
3.3
Tracking Output Consensus
A more general problem known as output consensus is not longer restricted to states, but to a function of states to converge to the same arbitrary or desired value. For consensus and tracking consensus fitness func-tions were selected of the formFi(x) =−xi, but the same problem can be solved with population dynamics
for any decreasing functionF(x).
Then population dynamics guarantee convergence to a Nash equilibriumx∗ implyingFi(x) =Fj(x) for
all i, j ∈ S when game F(x) is stable. Fitness functions can be related to outputs of a system and it is desired that all outputs converge to a same reference value.
Consider a graphG(Figure 10) representing the topology of a society and that determines communication among strategies. Fitness functions for this society are:
F(x) =
−2x1+ 50
−2x2+ 40
−2x3+ 12
−2x4+ 30
−2x5+ 60
−2x6+ 70
−2x7+ 85
−2x8+ 20
−2x9+ 21
−2x10+ 23
(35)
1
2 3
r
4 5 6 7
8 9
10
Figure 10: Graph for output consensus problem example and Society graphG.
Additionally, it is desired that trajectories of fitness functions track a reference varying over time. Refer-ence signal has the sequRefer-ence [2,10,2,−10] fort <10, 10< t <20 and 20< t <30,30< trespectively. Due to the fact this is a control problem in which the output of systems track a reference, robustness is tested by adding critical noise inF1(x) whose amplitud is 15. Result for this tracking output consensus is presented
0 5 10 15 20 25 30 35 40 −30
−20 −10 0 10 20
30 Fitness functions
Time [seg]
Fi
(
x
)
w
it
h
1
≤
i
≤
10
Figure 11: Tracking output consensus.
4
Distributed Control Based on Population Dynamics for
Drink-ing Water Networks: Application Case Study.
The Barcelona DWN [16] is managed by Aguas de Barcelona S.A (AGBAR) and supplies drinking water to Barcelona city and also metropolitan area. Sources in the network are Ter and Llobregat rivers. There are four treatment plants (WTP) which are: Abrera, Sant Joan Despi, Cardedeu and Besos plants.
4.1
System management criteria
In order to control DWN system, there are some indices for a desired performance. There are three main parameters to design: i) operation cost associated to water costs and energy costs, ii) safety storage terms which are related to satisfy demand at each point in the network and iii) Smoothness of the control actions to avoid damage throughout the network.
Costs: Economic costs associated to drinking water producting are generated by chemicals for treatment water, legal canons and active elements electrical costs (pumps). The function to minimize is given by
f1(t) =Wc(α1+α2(t))u (36)
whereWc establishes a weight for a total cost function,u∈RN denotes the flows that can be manipulated,
α1is a known column vector whose cardinality isN associated to water costs according to the corresponding
source (treatment plant, dwell, etc.). α2(t) is a column vector whose cardinality is N describing operation
energy costs associated to pumps and that varies over time since electricity costs have differente value during the day.
Safety storage term: Satisfaction of water demands should be met all the time and some risk prevention mechanism should be considered as a storage of water volume availability of emergency for future demands given by a relation respect to the maximum volume at each tankξvmax. This function to minimize is
f2(t) = (v−ξvmax)>Wv(v−ξvmax) (37)
where v∈RV denotes the water volumes at network tanks, ξis a term that determines the safety volume
to be considered respect to the maximum volume tank vectorvmax. Wv is a weight matrix to ponderate a
total objective function.
Smoothness of control actions: For a normal operation, the variations of control actions (manipulated flows) should be smooth, avoiding overpressures which can cause structural damage and leaks in the network. This factor is going to be considered in the control design but not explicitely in the objective function, however population dynamics are design to evolutionate slowly avoiding hard changes in population proportions. The
function that penalizes these abrupt changes inuis
f3(t) =
du(t) dt
>
Wu
du(t) dt
(38)
whereWu is a weight matrix N×N.
4.2
DWN Model and constraints
The drinking water network is a large-scale system. Figure 12 shows the DWN in Barcelona. Control system presented in this paper manages the transport network problem and sets references for local PID controllers which are in a lower level of the hirarchical structure. DWN is a non-linear system, however for manage-ment level it is considered linear due to the fact that non-linearities are considered in the low level controllers. The mass balance expression relating the stored volumev, the manipulated flows and demand flows can be written as the following equation
˙ vi =
X
n
qin,n(t)−
X
m
qout,m(t) (39)
For all the timeq(in),n(t) andq(out),m(t) are thenth input flow and themthoutput flow to theith tank
respectively. These flow are given in [m3/s]. There are physical constraints for the tanks according to their
capacity.
vimin≤vi(t)≤vimax (40)
where vmin
i is the minimum volume capacity andvmaxi is the maximum capacity given in [m3]. Then,
it is not possible to asign more water to a fulled in tank or drain water from an empty tank. Regarding control actionesui(t), there are two types of actuators: pumps and valves. These elements have operation
constraints as well.
umini ≤ui(t)≤umaxi (41)
where umini and vmaxi denote the minimum and maximum flow capacity respectively given in [m3/s]. Figure 12 shows 11 nodes in the network where mass balance should be respected, adding more constraints to the problem. These contraints are expressed as
X
n
qin,n(t) =
X
m
qout,m(t) (42)
where qin,n(t) is the nthinput flow to the node and qout,m(t) is the mth output flow to the node. Some of
these flows are manipulated as part of control actions and others are set by demands of water. Demand can be forecasted and they are assumed to be known where dis a vector with measured disturbances affecting DWN. Nodes constraints are shown in table 1.
4.3
Optimization problem
Controller is designed as an optimization problem minimizing costs and the error respect to safety storage term subject to physical constraints of nodes, tanks and actuators. Then error is defined ase(t) = (ξ˜vmax−
˜
v)u(t) since it is known that error e is reduced as u is increased assigning more drinking water to tanks with more error. ˜v,˜vmax∈
RN andξ˜vmaxi −v˜i is the error of the tank for which ui is its input flow, and
ξ˜vmax
i −˜vi= 0 ifui is not an input flow for any tank, e.g., foru(1), u(2), u(14), ..., u(61). Additionally, it is
pointed out thatf2(t) is reduced ifeis reduced.
maximize f(x) =−Wc(α1+α2(t))u(t)−Wv(˜v−ξ˜vmax)>diag(u(t))(˜v−ξ˜vmax)
subject to Au=b
Table 1: Nodes constraints Node Constraint
1 u(1)−u(2)−u(5)−u(6) = 0 2 u(2)−u(3) =d(2)
3 u(18)−u(13) =d(5)
4 u(14) +u(26) +u(15)−u(19)−u(25) =d(7) 5 u(22)−u(30) =d(9)
6 u(31)−u(40)−u(39) =d(14)
7 u(32) +u(40) +u(34) +u(25)−u(41)−u(26) =d(15) 8 u(39) +u(46)−u(45)−u(47) =d(17)
9 u(28) +u(49)−u(35)−u(43) =d(16) 10 u(44) +u(43) +u(52) =d(20)
11 u(61)−u(59)−u(50)−u(51)−u(52)−u(57)−u(58)−u(56)−u(60) =d(25)
whereAa 11×61 matrix is determined by constraints presented in table 1 corresponding to nodes,Iis the identity matrix 61×61 fixing constraints related to actuators,b= 0 d(2) d(5) d(7) ... d(25) >
is a column vector 11×1 anddis a column vectorumax. This is the same optimization problem (30). Then,
consider the vector of states and slack variables ˜x= (u>, s>)T with dimension 122×1 and a new matrix ˜A
and ˜b
˜
A72×122=
A11×61 011×61
I61×61 I61×61
˜b72
×1=
b11×1
d61×1
(44) The new optimization problem is the form as in (23) and cen be solved using population dynamics whose fitnesses are the Lagrangian function.
maximize f(x) subject to A˜˜x= ˜b
˜
xi ≥0 where 1≤i≤122 (45)
4.3.1 Partition of the network
Partition of the network is a problem already studied [17], for this optimization problem partition is deter-mined based on the constraints. As in Section 2.4, Lagrange multiplier vertices are connected to vertices from which need information to set the respective constraint and those must belong to the same clique. Based on this idea, it is posible to determine vertices (strategies) that should belong to the same clique (population). As an example, constraint given by node 9 shares a vertex with constraint given by node 10, this vertex is u(43); and constraints for node 10 and 11 involve vertexu(52). Consequentely, there is a clique including all elements involved in constraints 9, 10 and 11 from table 1. In the same way, vertices corresponding to slack variables(j) should belong to the same clique to which vertexu(j) belongs, since there is a Lagrange multiplier connected to them.
On the other hand, there are some vertices which are not associated to a constraint, likeu(4) oru(55). In this case vertices are assigned to the nearest cliques, thenu(55) might be assigned to the same clique to whichu(54) belongs. Cliques are presented in table
4.4
Results
Simulation consider the following parameters. Demand requirements establishes a ξ= 0.2, and weight for functions are selected to assign more importance to the minimization of costs, thenWc= 10 andWv= 0.1.
Objective functionf(x) is shown in figure 13. Initial conditions for simulation is v(0) = 0.193vmax.
Results of drinking water network states of tanks 7 and 12 are shown in figure 14. Volumes oscilate around the set point to satisfy safety storage demand.
Table 2: Partition of the network into cliques
Clique Cp Vertices u, s(.)∈ Vp Amount |Vp|
Cliquep= 1 1,2,3,4,5,6,7,8,9,10,11,13,17,18,22,29,30,36,37,38 40 Cliquep= 2 12,14,15,16,19,20,21,23,24,25,26,27,31,32,33,34,39,40,41,45,46,47 44 Cliquep= 3 28,35,42,43,44,48,49,50,51,52,53,54,55,56,57,58,59,60,61 38
Figure 12: Drinking water network. Clique 1 (Blue), Clique 2 (Green) and Clique 3 (Red).
Regarding costs of water and energy costs related to actuators operation, data are discriminated in Table 4.4 during 5 days. These costs increase for an initial condition farther from set-point.
5
Conclusions
A new method to solve games in a distributed way has been presented. Under conditions of the graph and the fitnesses to construct a full potential game that is stable, optimization problems with multiple constraints are solved in a distributed way based on the idea to have dynamic population feasible regions given by masses. Additionally, the method allows to have modular networks in the sense that a modification on the graph implies a local modification and it is not necessary to design again the whole system.
As an advantage, the system to solve optimization problems is robust since trajectories always converge to feasible region in any case states are out of it. As a problem set-up, initial conditions are selected to be
0 20 40 60 80 100 120 2
2.2 2.4 2.6 2.8 3 3.2 3.4 3.6 3.8
4x 10
4 Objective function
Time [seg]
f
(
x
)
Figure 13: Objective functionf(x).
0 20 40 60 80 100 120
1.25 1.26 1.27 1.28 1.29 1.3 1.31 1.32x 10
4 Volume evolution in tank 7
Time [seg]
v7
[
m
3]
0 20 40 60 80 100 120
1.89 1.9 1.91 1.92 1.93 1.94 1.95 1.96 1.97x 10
4 Volume evolution in tank 12
Time [seg]
v1
2
[
m
3]
a) b)
Figure 14: Volume evolution with reference for tanks a) 7 and b) 12.
in the feasible region, however due to the fact this systems are proposed to be applied on-line, changes on the system or noise might occur and this could make states get outside the feasible region momentaneously. Moreover, the fact system guaratees convergece to feasible region, allows to apply the method to systems whose constraints vary over time. Regarding consensus, the method can be used to solve consensus prob-lems, tracking consensus and output tracking consensus. The main advantage of this method is in the case information has a delay. As shown in results, the delay for tracking consensus or output tracking consensus does not affect considerably the convergence.
Finally, but not less important, the method can be used for cases in which each clique evolutionates in a different rate time since dynamics are decouple as a game. Also, this method is used to control systems without having the model since it is enough by knowing the tendency of the system according to the control action associated to states in the game.
Table 3: Discrimination of economic costs Day Water Electric Total
1 26.44 18.17 44.61 2 20.13 11.4 31.53 3 11.83 10.2 22.08 4 9.981 7.877 17.858 5 10.54 9.013 19.553
References
[1] N. Li and J. R. Marden, “Designing games for distributed optimization,” IEEE Journal of Selected Topics in Signal Processing, vol. 7, no. 2, pp. 230–242, 2013, special issue on adaptation and learning over complex networks.
[2] A. Pantoja and N. Quijano., “Distributed optimization using population dynamics with a local replicator equation.” inConference on Decision and Control (CDC). IEEE, 2012, pp. 3790–3795.
[3] M. Mesbahi and M. Egerstedt,Graph theoretic methods in multiagent networks, ser. Princeton series in applied mathematics. Princeton: Princeton University Press, 2010.
[4] Y. Hatano and M. Mesbahi, “Agreement over random networks.”IEEE Trans. Automat. Contr., vol. 50, no. 11, pp. 1867–1872, 2005.
[5] W. H. Sandholm, Population games and evolutionary dynamics, ser. Economic learning and social evolution. Cambridge, Mass. MIT Press, 2010.
[6] S. Mannor and J. S. Shamma, “Multi-agent learning for engineers,” Artif. Intell., vol. 171, no. 7, pp. 417–422, May 2007.
[7] A. Pantoja and N. Quijano, “A population dynamics approach for the dispatch of distributed genera-tors,”IEEE Transactions on Industrial Electronics, vol. 58, no. 10, pp. 4559–4567, 2011.
[8] H. Khalil,Nonlinear Systems. Prentice Hall PTR, 2002.
[9] H. C. Johnston, “Cliques of a graph-variations on the bron-kerbosch algorithm.” International Journal of Parallel Programming, vol. 5, no. 3, pp. 209–238, 1976.
[10] I. M. Bomze, M. Pelillo, and V. Stix, “Approximating the maximum weight clique using replicator dynamics.”IEEE Trans. Neural Netw. Learning Syst., vol. 11, no. 6, pp. 1228–1241, 2000.
[11] J. Hofbauer and W. H. Sandholm, “Stable games and their dynamics,” Journal of Economic Theory, vol. 144, no. 4, pp. 1665–1693.e4, 2009.
[12] E. Chong and S. Zak,An Introduction to Optimization, ser. Wiley Series in Discrete Mathematics and Optimization. Wiley, 2011.
[13] J. Wang and N. Elia, “A control perspective for centralized and distributed convex optimization.” in CDC-ECE. IEEE, 2011, pp. 3800–3805.
[14] “Consensus tracking with a reference state,” in Distributed Consensus in Multi-vehicle Cooperative Control, ser. Communications and Control Engineering. Springer London, 2008, pp. 55–73.
[15] R. Olfati-saber, J. A. Fax, and R. M. Murray, “Consensus and cooperation in networked multi-agent systems,” inProceedings of the IEEE, 2007, p. 2007.
[16] C. Ocampo-Martinez, V. Fambrini, D. Barcelli, and V. Puig, “Model predictive control of drinking water networks: A hierarchical and decentralized approach,” inAmerican Control Conference (ACC), 2010. IEEE, 2010, pp. 3951–3956.
[17] C. Ocampo-Martinez, S. Bovo, and V. Puig, “Partitioning approach oriented to the decentralised pre-dictive control of large-scale systems,”Journal of Process Control, vol. 21, pp. 775–786, 2011.
