Televisión como un instrumento en la formación de la identidad

5.2.1 The Sobol’ variance decomposition method

A number of different global sensitivity analysis approaches have been proposed (Sheri- dan,2008). The Sobol’ variance decomposition (Sobol,1993) is chosen due to its effec- tiveness compared with other global sensitivity analysis approaches (Tang et al.,2007). The first-order effectSiand the total-order effectSiT are two widely used measures in the

Sobol’ variance decomposition method. Considering a modelY =f(x1, x2,· · ·, xh) that

hash parameters,Si measures the expected reduction in variance that can be achieved

if one input parameter, xi, is fixed. Meanwhile, SiT measures the remaining expected

variance if all parameters exceptxi are fixed.

Mathematically,Si is defined as follows (Hadka and Reed,2012):

Si =

Vxi[Ex∼i(Y|xi)]

V(Y) (5.1)

wherexi is theith parameter andx∼i represents a container that contains all the param-

eters except xi. Ex∼i(Y|xi) in the numerator represents the expected value ofY, taken

over all possible values of x∼i when xi is fixed. Vxi[Ex∼i(Y|xi)] measures the variance

of Ex∼i(Y|xi), taken over all possible values of xi.

Si only measures the effect of a single input parameter to the variance of the output.

It has not taken into account the interactions Vij between two input parameters. Vij is

defined as follows:

Vij =V(E[Y|xi, xj])−V(E[Y|xi])−V(E[Y|xj]) (5.2)

where V(E[Y|xi, xj]) describes the joint effect of the pair of parameters (xi, xj) on Y.

This is known as the second-order effect. Higher-order effects can be computed similarly, e.g. the variance of the third-order effect of three parametersxi,xj and xk is:

Vijk=V(E[Y|xi, xj, xk])−Vij −Vjk−Vik−Vi−Vj−Vk (5.3)

S_iT denotes the sum of the first and higher order effects of xi. Taking a model with

three input parameters (h = 3) as an example, the total order effect of x1 is S1T =

S1+S12+S13+S123, whereS12 and S13 are the second-order effect ofx1,S123 is the third-order effect of x1. Obviously, computing the total-order effect using Equation 5.3 is not appropriate when h is large. However, from Equation 5.1, S∼i can be extended

S∼i =

Vx∼i[Exi(Y|x∼i)]

V(Y) (5.4)

S∼i represents the first order effect of x∼i. Therefore, V(Y)−Vx∼i[Exi(Y|x∼i)] is the

contribution of all terms in the variance decomposition that includes xi (Saltelli et al.,

2010). Thus,S_iT can be computed by:

S_iT = 1−Vx∼i[Exi(Y|x∼i)]

V(Y) (5.5)

In practice,Si andS_iT can be computed by evaluating the model at a number of sample

points. These points can be selected by using the Monte Carlo method. Each point corresponds to a model output. In this study, all the Monte Carlo samples of the parameter space are generated using the Sobol’ quasi-random sequence sampling method (Sobol and Kucherenko,2005). Si and SiT are calculated by the approach presented in

Saltelli(2002) withq×(h+ 2) runs, whereh is the number of parameters, and q is the sample size which is set to 2h as suggested inSaltelli (2002).

To validate that the sensitivity results are caused by the effect of parametrisation rather than the stochastic effects, the bootstrap technique called the moment method is applied to evaluate the confidence level of the Sobol’ indices, producing 95% confidence intervals. The reason for choosing the moment method is that it produces reliable results using a small resampling size (Archer et al.,1997). In this study the resampling sizeB is set to 2 000 as suggested in Saltelli et al. (2010). Assuming that we have determined Si and

ST

i for each parameter of the model Y = f(x1, x2,· · ·, xh), the Bootstrap Confidence

Interval (BCI) for each parameter is computed as follows, takingSi as an example (BCI

forS_iT is calculated in a similar way):

(i) Resample B groups of the parameters, (x1, x1,· · ·, xh), with which we can then

obtain B outputs of Y. Si is recalculated at each sampling time. This produces a

bootstrap estimate of the sampling distribution of the sensitivity indices,S_ib, where b= 1,2,· · · , B.

(ii) BCI forSi with 95% confidence is then determined bySi±1.96×std(Si), where

std(Si) = q 1 B−1 PB b=1(Sib− 1 B PB b=1Sib) 2 .

When the value ofSi orSiT exceeds the BCIs of Si orSiT, we say the sensitivity indices

cannot be reliably computed. In other words, the variance of Si orSiT is mostly caused

by stochastic effects. More precisely, a large confidence level ofSiorSiT is due to the fact

that the effect of parameterization is not significantly stronger than stochastic effects (Hadka and Reed,2012).

5.2.2 Experiment description

There are h = 7 parameters in both PICEA-g and PICEA-w. These parameters are the number of function evaluations,NFS, the population size of candidate solutions,N, the number of preferences,Np, the crossover rate of SBX, pc, the mutation rate of PM,

pm, the SBX distribution index, ηc and the PM distribution index, ηm. The sampled

parameter space of these parameters is shown in Table5.1. ParameterNp is the number

of preferences used in the PICEAs. Np refers toNg in PICEA-g, and Nw in PICEA-w.

Table 5.1: The sampled parameter space. values NFS 2500 5000 10000 15000 20000 25000 30000 35000 40000 50000 N 20 50 100 150 200 300 400 500 700 1 000 Np 20 50 100 150 200 300 400 500 700 1 000 pc 0.01 0.05 0.1 0.2 0.4 0.5 0.7 0.8 0.9 1 ηc 1 5 10 20 50 100 200 300 400 500 pm 0.01 0.05 0.1 0.2 0.4 0.5 0.7 0.8 0.9 1 ηm 1 5 10 20 50 100 200 300 400 500

ParameterNFS is sampled from 2 500 to 50 000 times, and parametersN andNp range

from 20 to 1000. Such configurations aim to permit reasonable execution times, while providing meaningful results. The configuration of parameters pc and pm covers the

entire feasible range [0,1]. The distribution indices of SBX (ηc) and PM (ηm) are based

on the choice used by Purshouse and Fleming (2007).

As mentioned earlier, the sample sizeqis suggested to be 2h, that is,q = 27 = 128 groups of parameter settings are sampled. For each test problem, the testing algorithm is run q×(h+ 2) = 128×(7 + 2) = 1152 times, which correspondingly produces 1152 Pareto approximation sets. The HV metric is then calculated for each Pareto approximate set. Thus, the final results comprise of 1152 parameter settings and their corresponding end-of-run HV values. Additionally, for each problem the testing algorithm is run independently 31 times in order to facilitate a statistical analysis.

Test problems are chosen from the WFG test suite. Specifically, all the even-numbered WFG problems are invoked for 2-, 4- and 7-objective instances. For each test problem, the WFG position parameter and distance parameter are set to 18 and 14, that is, each problem has n= 18 + 14 = 32 decision variables.

5.2.3 Experiment results

Figure 5.1 shows the first-order (Si) and total-order (SiT) effects of the parameters in

PICEA-g in terms of the HV metric. Similarly, the results for PICEA-w are shown in Figure 5.2. Si represents the contribution of a single parameter to the variance of

2−2 4−2 6−2 8−2 2−4 4−4 6−4 8−4 2−7 4−7 6−7 8−7 η_m p m η_c p_c N N g NFS 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Less important More important

(a) the first-order effect

2−2 4−2 6−2 8−2 2−4 4−4 6−4 8−4 2−7 4−7 6−7 8−7 η_m p_m η_c p c N N_g NFS 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Less important More important

(b) the total-order effect

Figure 5.1: The sensitivity of an individual parameter in PICEA-g measured by the Sobol’ variance decomposition method.

2−2 4−2 6−2 8−2 2−4 4−4 6−4 8−4 2−7 4−7 6−7 8−7 η_m p m η_c p_c N N w NFS 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 More important Less important

(a) the first-order effect

2−2 4−2 6−2 8−2 2−4 4−4 6−4 8−4 2−7 4−7 6−7 8−7 η_m p_m η_c p c N N_w NFS 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Less important More important

(b) the total-order effect

Figure 5.2: The sensitivity of an individual parameter in PICEA-w measured by the Sobol’ variance decomposition method.

theHV distribution. In each figure, the shaded grid corresponds to a problem instance (x-axis) and one of the parameters (y-axis). For brevity in the x-axis label we usen−Y to denote the problem WFGn−Y. For example, 2-4 refers to WFG2 with 4 objectives. The intensity of the grid represents the importance of a parameter. A larger intensity indicates that the corresponding parameter has a significant effect on algorithm perfor- mance whereas a small intensity implies that the effect of changing the parameters is negligible. In Figures5.1and5.2, all the indices are reliable as the calculated indices are all within the 95% BCIs. This implies that the sampling size q= 2h_{= 128 is sufficient.}

Key observations are as follows:

(i) Combining both the first-order and the total-order effects, it is observed that the parameter NFS is the most significant parameter for both PICEAs across most problems. This indicates that PICEAs are user-friendly as the algorithm per- formance is controlled for the most part by a single parameter. Parameterising PICEAs should be easy in practice. A better performance can be obtained sim- ply by lengthening the runtime of PICEAs, i.e., increasing the number of function evaluations. Additionally, it is worth mentioning thatNFS is not always the most dominant parameter for all MOEAs. For example, it is demonstrated inHadka and Reed(2012) that in some MOEAs, such asε-MOEA (Deb et al.,2005) that utilises ε-dominance archives, the setting ofε-dominance is the most dominant component for controlling the algorithm performance.

(ii) Amongst the remaining six parameters, population size,N and preference size, Np

are relatively more significant to algorithm performance. As the number of objec- tives increases, these two parameters become more important. However, there is an exception, that is, the influence of Np is not significant on WFG2 problems.

One reason might be that the Pareto optimal front of WFG2 problem is discon- nected and only occupies a small portion of the objective-space. Therefore, a small number of preferences are enough to guide the solutions approximate the entire Pareto optimal front. Another reason might be that the reference point chosen for theHV calculation is not appropriate; it cannot distinguish the quality of different Pareto approximation sets. With respect to the four genetic operators, pm is the

most dominant, especially on the WFG2 problems.