Variables

sampling

As we lack mathematical tools to minimize the variance over the set of the possible importance densities, one often restricts the minimization to a set of parametric densities, which allows optimizing over a set of parameters. In the case where h = 1D, and Y involves an empirical mean of random variables the choice of the parametric densities can be based on a large deviation analysis, see [28, 29, 50], though this supposes that the random variable Y is defined in an Euclidean space. In other cases the parametric family is generally built using the practitioner’s knowledge about g∗_{, or, if no valuable knowledge}

2.3.1

Variance optimization: the Cross-Entropy method

Once the parametric family of importance densities is chosen, we can use different methods to find the best parameters. We denote by α ∈ Aparam ⊆ Rn the possible parameters, where n ∈ N∗ _{and A}

param is the set of the possible parameters. We denote by (gα)α∈Rn the parametric importance densities. In this subsection we present the Cross-

Entropy method [20, 34] which optimizes the parameters in order to get the best variance reduction.

The Cross-Entropy method consists firstly in measuring a distance between the den- sities gα and g∗, and secondly in minimizing it or in minimizing an approximation of this distance to get the best parameter. The distance between the densities gα and g∗ is measured thanks to the Kullback-Leibler pseudo-metric:

D(g∗, g) = Eg∗ " log g∗(Y ) g(Y ) !# . (2.46)

As Eg∗[log (g∗(Y ))] is a constant (i.e., it does not depend on g), minimizing

D(g∗, gα) = Eg∗[log (g∗(Y ))] − E_g∗[log (g_α(Y ))] (2.47)

is equivalent to maximizing

Eg∗[log (g_α(Y ))] , (2.48)

and as g∗_{(y) =} h(y)f (y)

p , minimizing D(g

∗_{, g}

α) is also equivalent to finding

α∗ ∈argmax α∈Aparam Ef " h(Y )loggα(Y )  # . (2.49)

This can be done if Ef

h

h(Y )loggα(Y )

i

and its derivative with respect to α can be computed analytically.

When Ef

h

h(Y )loggα(Y )

i

can not be computed analytically, one can approximate it empirically by using the following estimator based on NCE,0 simulations:

1 NCE,0 NCE,0 X i=1 h(Yi)log  gα(Yi)  , where Yi ∼ f. (2.50)

A first approximation of α∗_{, denoted by α}

1, can now be given by solving:

1 NCE,0 NCE,0 X i=1 h(Yi)∇αlog  gα(Yi)  = 0, where Yi ∼ f (2.51) Note that, in (2.50), it is important to get at least one realization Yi such that h(Yi) 6= 0, so that the minimization in (2.51) yields a meaningful minimum α1. If the practitioner

chose a number NCE,0, there is a chance that all simulations are such that h(Yi) = 0. Therefore in practice, it can be interesting to first chose a number nCE ∈ N∗ and to to gradually increase the number of simulations NCE,0 so that nCE of the simulations are such that h(Yi) 6= 0. With a well-chosen parametric family of importance densities this optimization should provide a parameter α1 for which the importance density gα1 is closer

from g∗ _{than f.}

Now note that, if we generate the simulations Yi’s according to the distribution gα1,

so it should provide a better approximation than in equation (2.50), because it is a better importance density than f in this case. By using an importance sampling strategy with the density galpha1, one can improve the approximation of Ef

h

h(Y )loggα(Y )

i

with the following estimator based on NCE,1 simulations:

1 NCE,1 NCE,1 X i=1 h(Yi) f(Yi) gα1(Yi) loggα(Yi)  , where Yi ∼ gα1. (2.52)

Consequently one can sometimes get a better parameter α2 by solving:

1 NCE,1 NCE,1 X i=1 h(Yi) f(Yi) gα1(Yi) ∇αlog  gα(Yi)  = 0, where Yi ∼ gα1. (2.53)

This step can even be iterated to get an α3, an α4, etc... This yields the algorithm 2.1,

where we denote f by gα0. It is also possible to start the algorithm 2.1 with gα0 6= f, in

order to start with a better empirical approximation of Ef

h

h(Y )loggα(Y )

i

.

Initialization: chose α0 ∈ Aparam and nCE ∈ N∗ and set t = 0, and ε > 0

while ||αk− αk+1|| < ε do

Set k = 1, and generate Y1 ∼ gαt while Pk i=11h(Yi)>0 < nCE do Generate Yk+1∼ gαt k := k + 1 N = k − 1 Compute αt+1= argmin α∈Aparam 1 N PN i=1h(Yi)_gf (Yi) αt(Yi)log  gα(Yi)  t := t + 1

End: Estimate p with ˆpαt−1

Table 2.1 – CE algorithm

2.3.2

Important remark on the Cross-Entropy

Note that the theoretical convergence properties of the CE method are not yet fully understood [40] and that the efficiency of this method has only been confirmed by many empirical examples. The convergence on the CE method has only been proven on par- ticular cases, and, in practice, there are cases for which the method does not converge.

This can be due to the fact that Ef

"

h(Y )loggα(Y )

 #

is poorly approximated because the number of simulations at each step NCE,t is too small, or because the approximation (2.52) is not convex in α and its minimization falls in a local minimum, or because the parametric family of candidate importance densities is ill-suited to the shape of h(Y )f(Y ). The approximation of Ef

"

h(Y )loggα(Y )

 #

is an importance sampling estimation so it suffers from the same practical problems which we mentioned in the subsection 2.2.5. This typically happens when h(Y )f(Y ) is multi-modal, which will often the case in the reliability assessment of a complex system.

It is therefore important NOT to blindly take the result of this method, and to check if its output is robust by running the method several times with different tuning parameters, and to check if the outputs are coherent with our knowledge of the shape of h(Y )f(Y ). Though they also do not guarantee the convergence, to go further one could also consider adaptation of the methods that have been proposed to handle multi-modal cases [33, 6, 26, 4].

2.3.3

Adaptive Cross-Entropy

We have seen previously that the CE method can be pursued if we can compute argmax α∈Aparam Ef " h(Y )loggα(Y )  #

analytically, or if we are able to approximate it which re- quires to generate a simulation such that h(Yi) 6= 0 at least once. For reliability assessment we have h = 1D and p = Ef[h(Y )] = P (Y ∈ D) is very low, so having h(Yi) 6= 0 (or equiv- alently Yi ∈ D) almost never happens. Indeed in practice the number of simulations to realize before getting Yi ∈ D is of the order 1/p, which is often too high compared to the computational capacities. To get the event h(Yi) 6= 0 in a reasonable time is not always possible. In this situation is it preferable to resort to a multi-level importance sampling technique, or an adaptive multi-level importance sampling technique. In this section we present one of these methods which is the adaptive Cross-Entropy method [20].

Assume we dispose of a real function S such that for s ∈ R:

p= P (Y ∈ D) = P (S(Y ) ≥ s). (2.54) The idea of the multi-level importance sampling is to decompose the problem into a sequence of intermediate problems of reduced complexity. That is to say where the event is obtained faster so that we can get good estimations faster. Let ˜s ∈ R, so that ˜s ≤ s. We denote ˜p = P (S(Y ) ≥ ˜s). The event S(Y ) ≥ ˜s contains the event S(Y ) ≥ s, so ˜p ≥ p. Therefore the event S(Y ) ≥ ˜s is easier to simulate.

The optimal density to estimate ˜p is given by:

∼

g∗(y) = 1(S(y)≥˜s)f(y)

˜p (2.55)

If ˜s is not too far from s the two densities∼

g∗and g∗ are very similar (they are proportional on D). Thus when a density gα gets close to

∼

g∗, it also gets close to g∗.

The technique of the multi-level importance sampling consists in taking a sequence of thresholds (st)t≥0for which, starting from a density gαt−1 tuned to estimate P (S(Y ) ≥ st−1),

one can easily find an efficient parametric density gαt to estimate P (S(Y ) ≥ st), and so

on.... If the sequence of thresholds tends to s one can construct a sequence of densities gαt that tends to gα∗ (the best density in the parametric family to estimate p).

This method works well when the thresholds (st)t≥0are chosen so that the probabilities Pαt−1(S(Y ) ≥ st) are not too big, about the order of ρ = 10

−2 _{for steps with 10000}

simulations, see [20]. Thus, in adaptive multi-level importance sampling stis chosen to be equal to the empirical quantile of order 1 − ρ.

In the end the adaptive multi-level importance sampling consists in applying the algo- rithm displayed in 2.2, where we denote by (S(1), . . . S(N )) an ordered sample of the S(Yi), so that the empirical quantile of order 1 − ρ of this sample is S(b(1−ρ)N c).

Initialization: choose α0 ∈ Aparam and ∀t, NCE,t ∈ N and set t = 0, s0 ≤ s

while st6= s do

Generate (Si)i = 1..NCE,t with Si = S(Yi) and Yi iid ∼ gαt Set st= min  S(b(1−ρ)NCE,tc), s  Compute αt+1= argmin α∈Aparam 1 N CE,t PNCE,t i=1 1S(Yi)>st f (Yi) gαt(Yi)log  gα(Yi)  t := t + 1

End: Estimate p with ˆpαt−1