Análisis de Cumplimiento de las Perspectivas para el Año 2015

Morris analysis was first taken in consideration as sensitivity method to perform group analysis, on all those test functions with a large number of variables. Successive testing of variance-based and Elementary Effects methods, to compare their performance on the test functions, highlighted a little gain using the second one in terms of compu- tational cost, both in the grouped and in single variables analysis, due to the simple process of evaluation. However, the first method can give further and more accurate information and so was chosen to describe all the test functions. Thus application of Morris method is only as mere comparison with results of the variance-based method to test group sensitivity analysis and concretely gets in touch with a screening method.

Variable µ∗ µ µ∗_{Conf. Lev.} σ

x1 307.35 307.35 16.91 194.18 x2 294.45 294.45 15.53 181.17 x3 15.84 0.83 1.72 25.45 x4 15.65 -1.33 1.65 24.62 x5 14.70 0.25 1.73 24.44 x6 16.20 0.13 1.60 24.57 x7 17.16 0.74 1.94 27.99 x8 16.30 0.59 1.66 24.92 x9 15.58 0.57 1.70 24.74 x10 15.49 0.10 1.82 25.37 . . . .

Table 6.3: Results of Morris sensitivity analysis for the first objective of DTLZ1, from appendix A.2 on test function. Only the first ten variables are reported here.

Figure 6.4: Histogram of Elementary Effects analysis referred to data in tab. 6.3.

Grouping 1 Grouping 2

Group Variables in the group

µ∗ µ∗_{Conf. L.} Group Variables in the group µ∗ µ∗_{Conf. L.} G1 x1, x2 398.76 11.35 G1 x1, x8 336.97 6.74 G2 x3, x4 21.70 1.30 G2 x2, x9 332.75 6.73 G3 x5, x6 20.17 0.94 G3 x3, x10 19.44 0.80 G4 x7, x8 20.11 0.96 G4 x4, x11 20.40 0.86 G5 x9, x10 20.39 1.00 G5 x5, x12 20.00 0.80 G6 x11, x12 20.25 0.96 G6 x6, x13 20.75 0.87 G7 x13, x14 19.72 0.96 G7 x7, x14 20.70 0.83

Table 6.4: Results of Morris sensitivity analysis by groups for the first objective of DTLZ1, from appendix A.2 on test function. Here there is a simple sequential coupling of the variables, but it is quite evident the difference on µ∗ value with respect to the previous table 6.3 in the first group.

The reported table 6.3 and 6.4 describe Elementary Effects results for DTLZ1 test function, defined by 14 variables, respectively considering each variable alone and grouping them in couples. Let’s briefly recall from section 5.3.2 the meaning of the calculated parameters: µ and µ∗ describe the overall mean influence of the selected decision variable on the objective function variation, respectively summing up each contribute with it sign and considering its absolute value; σ instead describe the vari- ance of the parameter µ, while µ∗_{Conf. Lev.} measure the confidence level of µ∗. Dealing with group sampling, it is useless measure each single influence with its sign and the relative variance, because it is function of two or more variables which has unknown behaviour taken singularly.

Looking at the analysis results and considering DTLZ1 test function A.2, in which the variables x1 and x2 brings always a positive contribute, it is quite outright find

this behave on the first table: both variables in fact present equal values of µ∗ and µ. Dealing instead with the other variables, one would immediately notice how µ parameter sets around zero and this, again, could be guessed a priori from the test function. Analysing finally the variability introduced in the objective function by

79 6.1. Sensitivity analysis application

Figure 6.5: Histogram of Elementary Effects analysis by groups referred to data in tab. 6.4.

each single variable it is clear from µ∗ values that first and second variables play a dominant role, as resulted also from the previous Sobol’ analysis. The σ parameter then describes how much it is the variance of each variable; it could further help to recognize the influence of the relative variable on the objective function, since variance of x1 and x2 introduce in the system much more fluctuation than the whole contributes

of the other variables.

Regarding the group analysis results, they confirm and strengthen the observation just made, though not bringing that much information by its own. However, looking at a single group analysis alone, Grouping 1, one would immediately obtain the most important information, which is the almost complete dependence of objective function from variables x1 and x2.

Moreover, in table 6.4 are reported two different grouping sample, both generated by couple of variables. This example well describes which are the main features of the screening method coupled with groups: first of all, this gives a fair view over the contribution of variables to the objective function as said above; furthermore, performing rearrangement of variables in the groups it is possible to deeply investigate the contribution of each single variable with lower computational costs. Here, with the first grouping, one would not know if just one or both variables x1 and x2 have large

influence on the function, but would immediately get that all the other variables are quite negligible. A second run with a new grouping is very important, because it is now possible to realize that both x1 and x2 are influential variables and with a similar

weight, matching the results. Developing a reasoned grouping sample for much larger and complex problem, e.g. with 500 or more variables and computational demanding objective functions, can provide a complete overview on the behave of the function, without requiring too much resources. Note finally that in this method does not need to realize groups of the same size necessary: once identified which could be the main variables, some groups could contain just each of them and a further group contains all the other negligible factors.

Figure 6.6 reports, as for the variance-based method, the change on computational time and confidence interval versus the sample dimension. However, in this case per- forming the analysis for single variables and grouping them would bias the results, because also the µ∗ value considered changes, therefore the relative mean value is re- ported in the figure description. As one could expect, the two behaviours obtained are

Figure 6.6: Variability of Confidence Level (upper) and computational time (lower) versus sample dimension for DTLZ6 problem. Average values of µ∗ for single variable analysis, group of 5 and 20 variables are respectively: µ∗₁ = 6.22, µ∗₃ = 0.11, µ∗_{group 1−5} = 8.38, µ∗_{group 1−20} = 8.64 and µ∗_{group 21−40}= 0.48.

very much similar to those observed above in the Sobol’ analysis, even if here the test problem used is the first objective function of DTLZ6 A.2, defined by 100 variables, be- cause it is much more suitable to several different grouping sizes. Observing the picture it is possible to notice a pretty large step in confidence level between singular variable and group analysis, which is present due to the chosen group selected: while groups which contain variables x1 and x2 grab also their large variable behaviour, groups that

involve only negligible factors would show their limited variability and therefore a con- fidence level almost null, as happen to the confidence level of a single variable alone x3.

Finally, observing grouping of the first 5 and 20 variables, the differences are almost null just because the factors, which are not x1 and x2, would bring an almost null contribute.

Dealing with the coding aspects, this part of the implemented model has been realized using the Sensitivity Analysis Library (SALib) available in the open source programming language Python [24]. Previously the use of this library, several routines dealings with Variance-based sensitivity methods were written, based on the articles of Sobol’ and Saltelli [52, 53, 56] as the library itself. Although, lacking the complete knowledge to evaluate the confidence level and some notion to realize the Sobol’ se- quence, briefly explained in [31], it has been chosen to use the library. On the other