El modelo de desarrollo

Last time an algorithm to compute the largest controlled invariant set contained in a given set F was introduced:

Algorithm 7.0 (Controlled Invariant Set) Initialization:

W⁰= F , W¹ =∅, i = 0 while Wⁱ = Wⁱ⁺¹ do begin

Wⁱ⁻¹= Wⁱ\ Reach(Pred(Wⁱ), Pre_u(Wⁱ)) i = i− 1

end

where the index decreases as a reminder that the algorithm involves a predecessor operation.

Recall that Pre_d(Wⁱ) is the set of states that are either already in the complement of Wⁱ, or may find themselves in the complement of Wⁱ through a discrete transition, whatever u does through an appropriate choice of d. Preu(Wⁱ), on the other hand, is the set of states that are already in Wⁱand there exists a u that can force a transition to another state in Wⁱ, whatever d does. Finally, Reach(Pred(Wⁱ), Preu(Wⁱ)) is the set of states that whatever u does there exists an appropriate response by d that drives the system to Pre_d(Wⁱ) without going through Pre_u(Wⁱ) along continuous evolution.

7.2 Computation

This is a pseudo-algorithm, in the sense that all the operators used were defined axiomati-cally. Therefore, even though conceptually the algorithm will produce the largest controlled invariant set, it can not be implemented as it is in practice. To effectively implement it and guarantee termination we need to:

1. be able to store arbitrary sets of states and perform operations on them.

2. compute Pre_u and Pre_d at each step.

3. compute Reach at each step.

4. ensure that only a finite number of steps are needed.

Finite sets can be stored and manipulated by enumeration. The situation is not as simple for continuous quantities however. Here we will assume that sets of states are stored as level sets of functions. This representation is convenient conceptually, but is not necessarily effective computationally, as it shifts the problem from storing sets to storing functions! For certain classes of sets/functions computation is in fact possible (for example, for polygons, ellipsoids, polynomials, functions encoded by neural networks, etc.). In this lecture we follow [77].

The computation of Pre_u and Pre_d is straightforward conceptually, as it involves inverting the discrete transition relation. The way to implement it computationally depends on the representation one uses to store and manipulate sets, and the sets that encode the discrete transitions (invariants, guards, reset relation). For certain classes (e.g. linear maps, polygonal sets, etc.) the computation may be feasible using quantifier elimination or optimal control tools.

Obtaining a representation of Reach is not only computationally involved, but also concep-tually. The standard optimal control and game theory concepts do not deal with this type of problem, where a certain set must be reached before another. Since q remains constant along continuous evolution, the computation of Reach can be performed separately for each value of q∈ Q. The computation will involve a modification of the Hamilton-Jacobi partial differential equation. Freeze the value of q and consider two sets K ⊆ X and L ⊆ X.

Assume that:

Assumption 7.1 Assume L is closed, K is open and there exist continuously differentiable functions l_L : X → [0, 1] and lK : X → [0, 1] such that lK(x) = 0 for x ∈ L, lK > 0 for x ∈ L and lK(x) = 1 for x ∈ K, whereas lL(x) = 0 for x ∈ K, lL > 0 for x ∈ K and lL(x) = 1 for x∈ L.

This assumption is satisfied if, for example, the sets have smooth boundaries, non-empty interiors and their closures are disjoint.

Consider the value function J :Rⁿ× R₋→ R governed by the Hamilton-Jacobi equation:

−^∂J_∂t^(x,t) =

The situation J (x, t)≤ −1 corresponds to points for which whatever u does, there exists a response for d such that the state reaches K by time |t|, without going through L for any 0≤ s < |t|. The situation J(x, t) ≥ 1 corresponds to points for which there exists a choice for u such that whatever d does the state reaches L by time |t| without going through K for any 0 ≤ s < |t|. Finally, the situation J(x, t) ∈ (−1, 1) correspond to states that are still “undecided” at time t: they may end up in K or in L by some time |s| > |t| or they may never reach either set. Notice that the situation is not quite symmetric because of the order of the quantification. As usual we are looking for a stationary solution to this PDE.

Proposition 7.2 If a continuously differentiable solution to the above equations exists that converges to a continuously differentiable J^∗ : Rⁿ → R as t → −∞, then Reach(K, L) = {x ∈ Rⁿ|J^∗(x) <−1}.

In the limit, the set where J^∗(x) <−1 consists of states that can ultimately be driven to K without first going through L, the set where J^∗(x)≥ 1 consists of states that can ultimately be driven to L without first going through K, while the set of states where J^∗(x)∈ [−1, 1) contains states that can not be forced to reach neither K nor L.

An alternative characterization of Reach(K, L) involves two coupled partial differential equations. Assume continuously differentiable function l_K : X→ R and lL : X→ R exists such that K = {x ∈ X|lK(x) < 0} and L = {x ∈ XC|lL(x) ≤ 0}. Consider the following Equation (7.2) describes the evolution of the set K under the Hamiltonian H_K^∗. This is the solution to the “maxumind” game for reachability in purely continuous systems, with the modification that H_K^∗ = 0 in {x ∈ X_C | JL(x, t) ≤ 0}. This ensures that the evolution of J_K(x, t) is frozen once this set is reached. Similarly, equation (7.3) describes the evolution of the set L under the Hamiltonian H_L^∗. Here a “minumaxd” is used, since it is assumed that the control tries to push the system into L, to escape from K. H_L^∗ = 0 in {x ∈ X_C | JK(x, t) ≤ 0} to ensure that the evolution of JL(x, t) is frozen once this set is reached. One can show that the resulting set {x ∈ XC | JK(x, t) < 0} contains neither L nor states for which there is a control which drives the system into L; and the set {x ∈ X_C | JL(x, t) < 0} contains neither K nor states for which there is a disturbance which drives the system into K.

Proposition 7.3 Assume that J_K(x, t) (J_L(x, t) respectively) satisfies the Hamilton-Jacobi equation (7.2) ((7.3) respectively), and that it converges as t → −∞ to a function J_K^∗(x) (J_L^∗(x) respectively). Then, Reach(K, L) ={x ∈ X_C| J_K^∗(x) < 0}.

As for the first representation of Reach, comments relating the values of J_K^∗ and J_L^∗ and the “fate” of the corresponding states can be made.

Remarks:

• The first representation is more compact. It involves solving a PDE in n + 1 dimen-sions, as opposed to the 2n + 1 dimensions needed for the second representation.

• The second representation is more general. For example, it does not require the closures of K and L to be disjoint. This often turns out to be needed in applications.

• In either case, we are still faced with the problem of finding a solution to PDEs.

Technical problems arise since a solution in the conventional sense does not always exist or may not be unique. For example, weaker solutions concepts (e.g. viscosity solutions) may be needed if the value function turns out not to be smooth.

• Even if we decide on an appropriate solution concept, obtaining the solution compu-tationally may still be a problem. Computation tools that may be applicable include:

– finite element methods – method of characteristics

– level set methods (these were studied in detail in the doctoral dissertation of Tomlin [79].

– approximate solution using basis functions – viability kernel computations

Each of these techniques has computational advantages and disadvantages and may be applicable in some problems but not others. They all suffer from the “curse of dimensionality”: they typically run out of steam beyondR³.

• Problems may arise even after a numerical solution is computed:

– will the numerical approximation be contained in the controlled invariant set?

– will the numerical approximation be controlled invariant itself?

– can a controller that renders the numerical approximation controlled invariant be synthesized?

– or at least, can a controller that renders the largest controlled invariant set invariant be synthesized using the approximation?