Descripción de las funciones de los puestos

This section outlines the experimental setup and MOGP evolutionary parameters, and presents an evaluation of the two diversity measures in the fitness function in MOGP (in terms of hyperarea of the evolved Pareto fronts and MOGP training times).

6.4.1

MOGP Setup and Evolutionary Parameters

The underlying approach to evolving accurate and diverse Pareto front solutions to represent the ensemble members uses MOGP with SPEA2 [188] (from the previous chapter). This means that the same MOGP framework is used to represent the genetic program solutions, and the evolutionary parameters in MOGP are also kept the same as the previous chapter. See the previous chapter for more details.

6.4.2

MOGP Pareto Front Hyperarea

To investigate how the diversity objectives in MOGP affects the performance of the evolved Pareto fronts (on the test sets), Table 6.1 reports the average (and standard deviation) hyperarea of the evolved Pareto-approximated fronts, and the hyperarea of the Pareto-optimal (PO) front for the NCL and PFC approaches (over 50 runs). Recall from the previous chapter that the PO front is the set of non- dominated solutions from the union of all Pareto-approximated fronts evolved over 50 runs. For a comparison, Table 6.1 also includes the average hyperarea and PO hyperarea of the Baseline MOGP approach, i.e., when no ensemble diversity objective is used in fitness. Note that the Baseline MOGP results are repeated here for convenience from Table 5.1 (in the previous chapter).

6.4. EVALUATION OF DIVERSITY MEASURES IN MOGP 155 Table 6.1: Average (± standard deviation) hyperarea of evolved Pareto- approximated fronts, and hyperarea of the Pareto-optimal (PO) front for the three MOGP approaches (Baseline, NCL and PFC) over 50 runs. The pairs of hyperarea results in bold or italics denote that these two approaches achieve a statistically significantly better hyperarea than the remaining approach (but not each other). The highest PO front hyperarea from all three approaches is underlined.

Task Basline MOGP MOGP with NCL MOGP with PFC Average PO Front Average PO Front Average PO Front

Ion 0.848±0.041 0.992 0.849±0.039 0.981 0.828±0.032 0.982 Spt 0.732±0.032 0.971 0.733±0.031 0.964 0.719±0.025 0.964 Ped 0.902_±0.019 0.922 0.905_±0.011 0.926 0.883_±0.010 0.921 Yst1 0.793±0.009 0.931 0.795±0.010 0.922 0.774±0.009 0.923 Yst2 0.949±0.011 0.991 0.949±0.007 0.972 0.928±0.011 0.989 Bal 0.757±0.063 0.985 0.810±0.078 1.0 0.800±0.065 1.0

Tukey’s Honestly Significant Difference (HSD) [166] multiple comparisons test is used to find the statistically significant differences in the average hyperarea for the three MOGP approaches (over 50 runs). Recall that Tukey’s multiple comparisons test compares the hyperarea from each system to all others (on a run-by-run basis), and outputs a 95% confidence interval for each pairwise comparison between systems. In four tasks (Ion, Ped, Yst1and Yst2), the average

hyperarea for the Baseline and NCL approaches is not statistically significantly different from one another, but both are statistically significantly better than PFC (these are highlighted in bold in Table 6.1). In Bal, the average hyperarea for NCL and PFC are not statistically significantly different from each other, but both are statistically significantly better than the Baseline approach (these are italicised in Table 6.1). In the remaining task (Spt), the average hyperarea for all three MOGP approaches are not statistically significantly different from one another. Note that the highest PO hyperarea achieved by a given MOGP approach over all 50 runs is underlined in Table 6.1.

Table 6.1 shows that in four out of six tasks (in bold in Table 6.1), the PFC approach finds non-dominated solutions with lower classification accuracy on the two classes than both the Baseline and NCL approaches, as the average hyperarea for PFC is the lowest. However, the frontier solutions from the PFC approach may be less accurate but potentially more diverse in their outputs than the Baseline and NCL solutions, due to the selection bias in fitness for PFC (this is explored further in the next section). In Bal, the hyperarea for PFC and NCL is significantly better than the Baseline MOGP. This suggests that the PFC and NCL

Table 6.2: Average training times for the three MOGP approaches in seconds (s) or minutes (m) over 50 runs.

Task Baseline MOGP MOGP with NCL MOGP with PFC Ion 9.3s±2.4 1.4m±6.8 30.4s±2.6 Spt 9.7s±2.5 1.2m±5.3 29.5s±1.6 Ped 3.9m_±1.1 90.2m_±1.3 17.7m_±24.8 Yst1 20.8s±7.1 5.6m±14.4 1.2m±4.5 Yst2 20.1s±8.1 5.3m±18.9 1.1m±5.9 Bal 15.2s±3.9 2.4m±8.5 45.1s±3.2

approaches find frontier solutions that are more accurate on the two classes, and also potentially more diverse in their outputs, than the Baseline MOGP.

In terms of the PO hyperarea for the three MOGP approaches (Baseline, NCL and PFC), the Baseline achieves the the highest PO hyperarea in four tasks (Ion, Spt, Yst1 and Yst2). As discussed above, this is due to the selection bias in the

Baseline MOGP where the PO solutions for NCL and PFC are less accurate (but potentially more diverse in their outputs) than the Baseline MOGP. In Ped, NCL achieves the best PO hyperarea; while both NLC and PFC achieve the best PO hyperarea in Bal. In Bal in particular, both NCL and PFC find at least one non- dominated solution with 100% accuracy on both the minority and the majority classes (on the test set). This solution represents the best PO hyperarea of 1 in Table 6.1 where the PO frontier consists of this one point alone (in objective- space). It is interesting that neither the Baseline MOGP approach, nor the single- objective GP methods using the different fitness functions (from Chapter 4), is able to accomplish this in any task. This suggests that identifying and promoting solutions with good accuracyanddiversity on the two classes using the NCL and PFC measures in the fitness function has the potential to achieve perfect solutions on difficult tasks such as Bal.

In Spt, the average hyperarea for all three MOGP approaches are similar to one another (i.e. not statistically different). Further analysis of the results for Spt finds that this is because the Pareto fronts for the three MOGP approaches tend to dominate each other in different regions of the objective-space. This can be seen later in Figure 6.4 which compares the median attainment summary surface for the PFC and Baseline MOGP approaches.

To compare the training times for the different MOGP approaches, Table 6.2 reports the average training times in seconds (s) or minutes (m) over 50 runs. Table 6.2 shows that as expected, both NCL and PFC incur longer average training times than the Baseline approach. This is due to the additional

6.5. MOGP ENSEMBLE CLASSIFICATION RESULTS 157