Principios del Coaching

We discuss how to compute the maximum probability to visit a setG ⊆ S of goal states during a given time intervalI. Therefore, letI be the set of nonempty intervals over the nonnegative reals and letQ be the set of nonempty intervals with nonnegative rational bounds. For t ∈ R≥0 and I ∈ I, we define I ⊖ t = {x − t ∣ x ∈ I ∧ x ≥ t} and I ⊕ t = {x + t ∣ x ∈ I}. Obviously, if I ∈ Q and t ∈ Q^≥0, this impliesI ⊖ t ∈Q and I ⊕ t ∈ Q.

6.2.1 A fixed point characterization for IMCs

LetM be an IMC. For a time interval I ∈ I and a set G ⊆ S of goal states, we define the event ◇^IG ={π ∈ Paths^ω∣ ∃t ∈ I. ∃s^′∈ π@t. s^′∈G} as the set of all paths that hit a state inG during time interval I. The maximum probability induced by ◇^IG inM is denoted p^M_max(s, I). Formally, it is obtained by the supremum under all GM schedulers:

p^M_max(s, I) = sup

D∈GM

Pr^ω_νs,D(◇^IG). (6.2)

For a schedulerD ∈ GM, s ∈S and interval I ∈ I with inf I = a and sup I = b, consider the functionsPr_ν^ω_s,D(◇^I⊖[⋅]G) ∶ t ↦ Pr^ωνs,D(◇^I⊖tG). Then Pr^ωνs,D(◇^I⊖[⋅]G) is piecewise continuous in R≥0by definition. As the following lemma proves, continuity (and thereby measurability) extends top^M_max(s, I ⊖ [⋅]):

Lemma 6.1 (Continuity of p^M_max). LetM =(S, Act, IT, MT, ν) be an IMC, G ⊆ S a set of goal states and I ∈I an interval. The functions p^M_max(s, I ⊖ [⋅]) ∶ R^≥0 → [0, 1] ∶ t ↦ p^M_max(s, I ⊖ t) are piecewise continuous and measurable for all s ∈ S.

Proof. For continuity, we prove that for all s ∈S and t ∈(R>0∖ inf I) it holds that

δ→0lim⁺p^M_max(s, I ⊖ (t − δ)) = p^Mmax(s, I ⊖ t) = lim

δ→0⁺p^M_max(s, I ⊖ (t + δ)). (6.3) Observe thatt = 0 and t = inf I are the only discontinuities of Pr^ω_νs,D(◇^I⊖tG): To see this, note that 0 ∉I ⊖ t for t < inf I and 0 ∈ I ⊖ t for t > inf I. Hence, if t = inf I, interactive transitions may reach a goal state directly without requiring integration over the time domain.

Further, observe thatPr^ω_νs,D(s, I ⊖ t^′) ≤ p^Mmax(s, I ⊖ t^′) for all t^′ ∈ R≥0by definition of p^M_max. To prove thatp^M_max(s, I⊖[⋅]) is piecewise continuous, we proceed by contraposition and assume there existst ∈(R>0∖ inf I) such that Eq. (6.3) is violated: Here we consider left-continuity and distinguish two cases: Assume thatp^M_max(s, I ⊖ [⋅]) is not continuous from the left at pointt ∈ R≥0and that there existsε > 0 such that

δ→0lim⁺p^M_max(s, I ⊖ (t − δ)) = p^Mmax(s, I ⊖ t) − ε. (6.4) Now, chooseD ∈ GM such that p^M_max(s, I ⊖ t) − Pr^ωνs,D(◇^I⊖tG) = ξ for some ξ ≤ ^ε₂. Then

p^M_max(s, I ⊖ t) − ξ = Pr^ωνs,D(◇^I⊖tG) = lim

δ→0⁺Pr^ω_νs,D(◇^I⊖(t−δ)G)

≤ lim

δ→0⁺p^M_max(s, I ⊖ (t − δ)).

But then, lim_δ→0⁺ p^M_max(s, I⊖(t−δ)) ≥ p^Mmax(s, I⊖t)−ξ > p^Mmax(s, I⊖t)−ε, contradicting Eq. (6.4). For the second case, assume that left-continuity att is violated because there existsε > 0 such that

δ→0lim⁺p^M_max(s, I ⊖ (t − δ)) = p^Mmax(s, I ⊖ t) + ε. (6.5) ChooseD ∈ GM such that limδ→0⁺Pr^ω_νs,D(◇^I⊖(t−δ)) = lim^δ→0+ p^M_max(s, I ⊖ (t − δ)) − ξ for someξ ≤ ^ε₂. Then

p^M_max(s, I ⊖ t) ≥ Pr^ωνs,D(◇^I⊖tG) = lim

δ→0⁺Pr^ω_νs,D(◇^I⊖(t−δ)G)

= lim

δ→0⁺p^M_max(s, I ⊖ (t − δ)) − ξ.

But then, limδ→0⁺ p^M_max(s, I ⊖(t−δ)) ≤ p^Mmax(s, I ⊖t)+ξ < p^Mmax(s, I ⊖t)+є, contradicting Eq. (6.5). Thus, p^M_max(s, I ⊖ [⋅]) is piecewise left-continuous. The fact that it is piecewise right-continuous follows along the same lines. Hence,p^M_max(s, I ⊖[⋅]) is piecewise contin-uous. As piecewise continuous functions are Borel measurable [Ros00, Prop. 3.1.8], we

are done. ◻

Based on the measurability of p^M_max(s, I ⊖ [⋅]), we are now ready to derive a fixed point characterization of the maximum probabilityp^M_max(s, I). More specifically, we prove that p^M_maxis the least fixed-point of a higher-order operator Ω:

Theorem 6.1 (Fixed point characterization for IMCs). Let M = (S, Act, IT, PT, ν) be an IMC, G ⊆ S a set of goal states and I ∈ I a time interval with inf I = a and supI = b for some a, b ∈ R≥0. The function p^M_max∶S × I →[0, 1] is the least fixed point of the higher-order operator Ω ∶(S × I → [0, 1]) → (S × I → [0, 1]), which is defined as follows:

1. For Markovian states s ∈ MS:

Ω(F)(s, I) =⎧⎪⎪

⎨⎪⎪⎩

∫₀^bE(s)e^−E(s)t⋅∑^s′∈SP(s, s^′) ⋅ F(s^′,I ⊖ t) dt if s ∉ G e^−E(s)a+∫₀^aE(s)e^−E(s)t⋅∑^s′∈SP(s, s^′) ⋅ F(s^′,I ⊖ t) dt if s ∈ G.

2. For interactive states s ∈ IS:

Ω(F)(s, I) =⎧⎪⎪

⎨⎪⎪⎩

1 if s ∈ G and 0 ∈ I,

max{F(s^′,I) ∣ s^′∈postⁱ(s)} otherwise.

Proof. The proof is split in two parts: First, we prove that p^M_max is a fixed point of Ω and second, we show that it is the least fixed point.

Recall that in Eq. (6.2) we defined p^M_max(s, I) = supD∈GMPr^ω_νs,D(◇^IG). To prove that p^M_maxis a fixed point of Ω, we first provide a disjoint decomposition of the event ◇^IG: Let γ(π, n) be the time interval which is spent in the n-th state of path π, measured in abso-lute time. Formally,γ(π, n) = [∆(π, n), ∆(π, n+1)) if ∆(π, n) < ∆(π, n+1) and γ(π, n) = {∆(π, n)}, otherwise. Now define the set Γ(I, n) of all paths whose (n+1)-th state is in G and lies within time intervalI, that is, Γ(I, n) = {π ∈ Paths^ω∣ π[n] ∈ G ∧ γ(π, n) ∩ I /= ∅}.

To achieve a disjoint decomposition of ◇^IG, set Π(I, n) = Γ(I, n) ∖ ⋃ⁿ⁻¹k=0Γ(I, k). Then

◇^IG =⊍^∞n=0Π(I, n). For D ∈ GM it holds:

Pr_ν,D^ω (◇^IG) = Pr^ων,D(⊍^∞

n=0

Π(I, n)) =∑^∞

n=0

Pr^ω_ν,D(Π(I, n)).

Further, letp^M,nmax(s, I) = supD∈GMPr_ν^ω_s,D(⊍ⁿi=0Π(I, i)) be the upper bound on the prob-ability to visitG during time interval I and within at most n transitions. First, we show thatp^M,n+1max (s, I) = Ω(p^M,nmax)(s, I). It suffices to consider two cases:

1. Lets ∈ MS and assume that s ∉ G (the case s ∈ G follows similarly). Then:

Ω(p^M,nmax)(s, I) =

∫

₀^{bE(s)e−E(s)t⋅}_s^∑′∈S

P(s, s^′) ⋅ p^M,nmax(s^′,I ⊖ t) dt

=

∫

₀^{bE(s)e−E(s)t⋅}_s^∑′∈S decisions for historyπ as the original scheduler D does for the history s Ð→ π,^{σ ,t} where we defines Ð→ π = s^{σ ,t} Ð→ s^{σ ,t} 0 shift allows us to rewrite Ω(p^M,nmax)(s, I) further:

Ω(p^M,nmax)(s, I) = sup

be another fixed point of Ω. By induction on the number of (interactive and Markovian) transitionsn, we show that p^M,nmax(s, I) ≤ F(s, I) for all n ∈ N.

1. In the induction base, it holds that p^M,0max(s, I) = 1 = Ω(F(s, I)) = F(s, I) if s ∈ G anda = 0; otherwise p⁰_max(s, I) = 0 ≤ F(s, I).

2. For the induction step, we distinguish between Markovian and interactive states:

(a) Lets ∈ MS and s/∈ G (the case s ∈ G can be shown similarly). Then p^M,n+1_max (s, I) = Ω(p^M,nmax)(s, I)

=

∫

₀^{bE(s)e−E(s)t⋅}_s^∑′∈S

P(s, s^′) ⋅ p^M,nmax(s^′,I ⊖ t) dt

≤

∫

₀^{bE(s)e−E(s)t⋅}_s^∑′∈S

P(s, s^′) ⋅ F(s^′,I ⊖ t) dt (* ind. hyp. *)

= Ω(F(s, I)) = F(s, I). (*F is fixed point *) (b) Now let s ∈ IS. If s ∈ G and 0 ∈ I, we have Ω(F)(s, I) = F(s, I) = 1 ≥

p^M,n+1max (s, I). Otherwise, the induction hypothesis yields

p^M,n+1_max (s, I) = Ω(p^M,nmax)(s, I) = maxs^′∈postⁱ(s)p^M,n_max(s^′,I) ≤ maxs^′∈postⁱ(s)F(s^′,I).

By definition of Ω, we havemax_s^′_∈postⁱ_(s)F(s^′,I) = Ω(F)(s, I) = F(s, I), prov-ing that p^M,n+1max (s, I) ≤ F(s, I).

Hence,F(s, I) ≥ limn→∞p^M,nmax(s, I) = p^Mmax(s, I) and the claim follows. ◻ Example 6.5. The fixed point characterization suggests to compute p^M_max(s, I) analytically:

Consider the IMC M depicted in Fig. 6.1 and assume that G = {s3}. For I = [0, b], b > 0 we have p^M_max(s3,I) = 1 and p^Mmax(s4,I) = 1 − e^−0.1b. For state s1, we derive that p^M_max(s1,I) = ∫₀^be−t(²₅ ⋅ p^M_max(s2,I ⊖ t) +₅¹⋅ p^M_max(s3,I ⊖ t) +²₅ ⋅ p^M_max(s4,I ⊖ t)) dt. In in-teractive state s2, we obtain that p^M_max(s2,I) = max {p^Mmax(s4,I), p^Mmax(s1,I)}, which yields that p^M_max(s0,I) =∫₀^{b0.9e−0.9t⋅}(²₃ ⋅ p^M_max(s1,I ⊖ t) +¹₃⋅ p^M_max(s2,I ⊖ t)) dt. ♢ From this example, it is easy to see that an IMC generally induces an integral equation system over the maximum over functions, which is not tractable. Moreover, the iterated integrations that occur are known to be numerically unstable [BHHK03].

Therefore, we resort to a discretization approach: Informally, we divide the time hori-zon into small time slices. Then we consider IPCs as a discrete-time model which we define such that its steps correspond to the IMC’s behavior during a single time slice.

First, we develop a fixed-point characterization for step bounded reachability in IPCs.

Then we reduce the maximum time interval bounded reachability problem in IMCs to the step interval bounded reachability problem in the discretized IPC. Finally, we show how to solve the latter by a modified value iteration algorithm.

6.2.2 A fixed point characterization for IPCs

Similar to the timed paths in IMCs, we defineπ@n ∈S^∗∪S^ωfor the time abstract paths in IPCs: Let #^PS(π, k) = ∣{i < k ∣ π[i] ∈ PS}∣; then #^PS(π, k) is the number of probabilistic transitions that complete up to the (k+1)-th state on π. For fixed n ∈ N, let i be the

smallest index such thatn = #^PS(π, i). If no such i exists, we set π@n = ⟨⟩; otherwise i is the index of the state that is reached on path π directly after the n-th probabilistic transition executed (or the first state onπ, if n = 0). Similarly, let j ∈ N be the largest index (or +∞, if no such finite index exists) such that n = #^PS(π, j). Then j denotes the position of the(n+1)-th probabilistic state on π. With these preliminaries, we define π@n =⟨si,si+1, . . . ,sj−1,sj⟩ to denote the state sequence after the n-th and up to the (n+1)-th probabilistic state ofπ. Intuitively, π@n is the sequence of states which are traversed during the(n+1)-th discrete time unit.

To define step-interval bounded reachability in an IPCP, let [ka,kb] ⊆ N be a step interval. Then

◇^[ka,kb]G ={π ∈ Pathsabs^ω ∣ ∃n ∈ {ka,ka+ 1, . . . , kb} . ∃s^′∈ π@n. s^′∈G}

is the set of paths that visitG between discrete time steps ka andkb inP. Accordingly, we define the maximum probability for the event ◇^[ka,kb]G:

p^P_max(s, [k^a,kb]) = sup

D∈GMabs

Pr^ω_νs,D(◇^[ka,kb]G).

Now, we are ready to provide a fixed point characterization for p^P_max:

Theorem 6.2 (Fixed point characterisation for IPCs). LetP = (S, Act, IT, PT, ν) be an IPC, G ⊆ S a set of goal states and I = [k^a,kb] a step interval. The function p^P_max is the least fixed point of the higher-order operator Ω ∶ (S × N × N → [0, 1]) → (S × N × N → [0, 1]) which is stated as follows:

1. For probabilistic states s ∈ PS:

Ω(F)(s, [ka,kb]) =⎧⎪⎪⎪⎪

⎨⎪⎪⎪⎪⎩

1 if s ∈ G ∧ ka = 0

0 if s ∉ G ∧ ka =kb= 0

∑s^′∈SPT(s, s^′) ⋅ F (s^′,[k^a,kb] ⊖ 1) otherwise.

2. For interactive states s ∈ IS:

Ω(F)(s, [ka,kb]) =⎧⎪⎪

⎨⎪⎪⎩

1 if s ∈ G and ka= 0

maxs^′∈postⁱ(s)F(s^′,[ka,kb]) otherwise.

Proof. The proof goes along the same lines as the proof of Thm. 6.1. First, we decompose the event ◇^[ka,kb]into disjoint subsets. Therefore, define

Γ([k^a,kb] , n) = {π ∈ Paths^ωabs∣ π[n] ∈ G ∧ k^a≤ #^PS(π, n) ≤ k^b} .

To achieve a disjoint decomposition of ◇^[ka,kb]G, we set Π([ka,kb], n) = Γ([ka,kb], n)∖

⋃ⁿ⁻¹i=0Γ([k^a,kb], i). Then Π([k^a,kb] , n) is the set of paths that visit G in the probabilistic step interval[ka,kb] for the first time after exactly n (probabilistic or interactive) transi-tions. Then ◇^[ka,kb]G =⊍^∞n=0Π([ka,kb] , n). Thus, it holds for all D ∈ GMabs:

be the upper bound on the probability to reach G during the probabilistic step inter-val[ka,kb] with at most n (interactive or probabilistic) transitions. Now we show that

p^P,n+1max (s, [ka,kb]) = Ω (p^P,nmax)(s, [ka,kb]):

In the remaining cases, we proceed as follows:

Ω(p^P,nmax)(s, [ka,kb]) = ∑

For the other cases, it holds that

It remains to show thatp^P_maxis the least fixed point of Ω: Thus, let F be another fixed point of Ω. By induction onn, we show that p^P,nmax(s, [ka,kb]) ≤ F(s, [ka,kb]):

1. For the base case,p^P,0max(s, [ka,kb]) = 1 = Ω(F(s, [ka,kb])) = F(s, [ka,kb]) if s ∈ G andka = 0 and p^P,0max(s, [ka,kb]) = 0 ≤ F(s, [k^a,kb]), otherwise. To see this, note that in the event Π([ka,kb] , 0) a G-state must be visited before any (probabilistic or interactive) transition executes.

2. For the induction step, we distinguish two cases:

(a) Lets ∈ PS: If s ∉ G (the case s ∈ G is similar), then

=maxs^′∈postⁱ(s)p^P,n_max(s^′,[ka,kb])

≤maxs^′∈postⁱ(s)F(s^′,[ka,kb]).

The definition of Ω implies max_s^′_∈postⁱ_(s)F(s^′,[k^a,kb]) = Ω(F)(s, [k^a,kb]) = F(s, [ka,kb]), proving that p^P,n+1max (s, [ka,kb]) ≤ F(s, [ka,kb]).

HenceF(s, [ka,kb]) ≥ limn→∞p^P,nmax(s, [ka,kb]) = p^Pmax(s, [ka,kb]), proving the claim. ◻ Observe the similarity in the treatment of interactive states in the fixed point charac-terizations for IMCs and IPCs: In an interactive state, the recursive expression of the time-interval bounded reachability in an IMC does not decrease the time intervalI for interactive states, whereas for IPCs, the recursive expression does not decrease the step interval[k^a,kb].

In this way, we have established a close relationship between IMCs and IPCs which allows us to discretize an IMC into an IPC. The details are the topic of the next section.

In document Propuesta de coaching para la reingeniería del proceso de elaboración de chocolates para la empresa el cacao. (página 30-33)