2.4. Marco Legal
2.4.2 Proceso de Trámite de importación a Brasil
The first measure to encourage diversity among the individuals in the population uses NCL as a correlation penalty term in the fitness function [40][36][35]. The NCL measure is based on the principle of minimisation of mutual information between variables [40][36][35]. In MOGP, NCL is used to measure the phenotypic differences between the solutions in the ensemble and the rest of the population. The NCL measure, given by Eq. (6.1), calculates the average correlation penalty for each class, for a given solutionpin the population.
N CLp = 1 2 K X c=1 1 M Nc Nc X i=1 (Gpi −Ei) " M X j=1,j6=p (Gji −Ei) #! (6.1)
6.2. MOGP APPROACHES FOR ENSEMBLE LEARNING 147 where
Gpi = 1 1 +egppi
In Eq. (6.1), K is the number of classes, and Nc is the number of training examples in class c. Gpi is the processed output, and gppi is raw output, of genetic program pwhen evaluated on the ith example in classc. The processed genetic program output is the raw program output when scaled between the range [0,1] using a transfer (or sigmoid) function. This is required by the NCL calculation, otherwise genetic programs which produce large output values risk unduly inflating the NCL penalty. The sigmoid function is applied to the value returned from the root node of the genetic program during the fitness evaluation. This ensures that positive raw output values are “spread out” between 0.5 and 1, and negative raw output values are spread out between 0 and 0.5 (when the raw output value is 0, the processed output is 0.5).
Ei is the output of the ensemble on the ith example in class c. This is the predicted class label returned by the ensemble, i.e., 0 or 1 for the majority or minority class, respectively, according to a majority vote of all ensemble members. The ensemble size (the number of non-dominated solutions in the current generation) is given byM.
As the NCL is a penalty term in fitness, the lower the NCL value for a particular class, the better the diversity of a solution.
In typical classification tasks where the class distributions are balanced, the NCL penalty is calculated with respect to all training examples [36][40][35][35]. However, Eq. (6.1) is adapted in this approach to calculate diversity separately for each class, to account for the skewed class distributions in these tasks. The average NCL penalty on the minority and majority then represents the final diversity estimate.
The NCL penalty is incorporated into MOGP by using Eq. (6.1) as the
secondaryfitness measure instead of the “crowding” distance. This means that the NCL term is used to resolve selection (e.g. for crossover/mutation and archive selection) when the primary fitness measure (Pareto ranking using SPEA2) is equal between two or more individuals. NCL is used as the secondary fitness measure because Eq. (6.1) requires the ensemble output (E) in its calculation. This means that the primary fitness measure must be applied to the population first to determine which solutions are non-dominated in the population (i.e. the current Pareto front), as these solutions form the ensemble.
Inputs 1 2 3 4 5 a1 0.8 0 0.7 0.5 0.2 a2 0.5 0.6 0.1 0.7 0 a3 0.1 0.7 0.6 0.1 1 E 1 1 1 1 0 (a) Calculation Inputs 1.a3−E -0.9 -0.3 -0.4 -0.9 0 2.a1−E -0.2 -1 -0.3 -0.5 -0.8 a2−E -0.5 -0.4 -0.9 -0.3 -1 (sum) -0.7 -1.4 -1.2 -0.8 -1.8 3. (1)×(2) 0.63 0.42 0.48 0.72 0 (b)
Figure 6.1: (a) The (processed) outputs for three solutions and the ensemble output (E) on the five inputs (incorrect predictions are underlined assuming that the target class label is1). (b) The three steps to calculate the NCL for solutiona3
where final NCL value fora3 is 0.15
P step 3 M×Nc = 2.25 3×5 . Example of NCL Calculation
To illustrate how NCL is calculated between solutions, Figure 6.1(a) shows the (processed) outputs of three genetic program solutions, a1, a2 and a3, on five
input examples from the minority class. Assuming that the target class label is 1(minority class), processed outputs that are ≥ 0.5 are considered correct class predictions; otherwise they are incorrect class predictions (these are underlined in Figure 6.1(a)). Assuming that these three solutions represent the ensemble, the ensemble output (i.e. predicted class label) is given by E in Figure 6.1(a). As discussed earlier, the ensemble output is obtained using a majority vote of the class labels of individual members. The ensemble outputE matches the target class label on the first four inputs, as exactly two individuals vote for the correct class label on these four inputs. Therefore, the ensemble accuracy in Figure 6.1(a) is 80% (four out of five inputs are correctly labeled), while each solution only achieves an individual accuracy of 60% (three out of five inputs correct).
Figure 6.1(b) shows how the NCL for solutiona3 is calculated on each input
instance. Step 1 corresponds to the first term in Eq. (6.1), i.e., Gpi −Ei, which compares the outputs of the given solution with the outputs of the ensemble. Step 2 corresponds to the second term in Eq. (6.1), i.e., PM
j=1,j6=pG j
i −Ei, which compares the outputs of the given solution with the other ensemble member’s outputs. In step 3, the results from the previous two steps are multiplied (for each input), and these values are then summed over the five inputs. This final value is then normalised (0.15) to represent the diversity for solutions3 on these
6.2. MOGP APPROACHES FOR ENSEMBLE LEARNING 149