The problem reported here is ZDT1. The response surface shows a good accuracy in fitting the true Pareto front, however it cannot overlap this last, as ASEMOO, GeDEA- II and Ge-DEA-II-K manage to do. Comparing in fig. 7.1 all the runs and the single run (respectively plots (a) and (b)), it is possible to highlight that each single run stands on the same line and does not vary a lot, as it happens for the previous three algorithms and not for the ParEGO. So the variability of results is pretty limited, as it can be seen both from the box-plots in fig. 7.2 and from the relative data in tab. 7.4 of normalized Hyper-Volume and D-metric. The D-metric box-plots show also that all the algorithms but the Response Surface have a little variability on results. This fact can be likely associated to the evolution searching process. Response surface is quite a deterministic algorithm and moreover in two-objective test functions the model involves just one variable to better fit the data. So its results cannot vary that much. Meanwhile, the other algorithms are much more based on the genetic optimization, which can produce quite different results at each run. This could lead a Pareto front always with a good Hyper-Volume value, but which presents also some element far from the true front. So its presence would make the D-metric change more than the HV at each run. In this case such behaviour is not evident because the best three algorithms always reach the true Pareto front without issues.
Let’s now give the results description for the other two-objective test functions, whose plots and data can be found in appendix B.
ZDT2 It presents a much higher accuracy of the results for the response surface method, as it can be seen in fig. B.1. This statement is even strengthened by the box-plots in fig. B.2. It shows that Hyper-Volume of the response surface as the other best ones approaches 1, while the D-metric measure is the lowest and so the best one, since it describes the overall proximity to the true Pareto front.
ZDT3 Conversely to the previous test, this problem shows a higher variability of the Pareto front. The response surface defined by a single variable cannot repro- duce faithfully this behaviour, but neither a response surface with more factors
Figure 7.1: Test function ZDT1: Pareto fronts for all runs (a) and single run (b) and for all the optimization algorithms.
ASEMOO GeDEAII GeDEAIIK RespSurf
10-1 100
Dmetric
ASEMOO GeDEAII GeDEAIIK RespSurf
0.8 0.85 0.9 0.95 1 HV norm
Figure 7.2: Test function ZDT1: box-convergence history of D-metric and normalized Hyper-Volume (with reference point at (1, 4)).
does. The figure B.1 and data reported displays the best parameter configuration possible, as it happens in the other cases. Even if comparison with the best algo- rithms shows a pretty worse result, the response surface performs anyway quite better than the ParEGO and it is quite near to the true Pareto Front. This fact is measured also in the normalized Hyper-Volume box-plot B.2, while dealing with D-metric it is evident the better results obtained by both ASEMOO and GeDEA-II-K.
ZDT4 In this successive problem there are reported two response surface curves in figure B.5. The best one (the lower, in yellow) refers to response surface with
93 7.3. Final results
D metric
ASEMOO GeDEA-II GeDEAII-K ParEGO RespSurf
Mean value 0.2177 0.2113 0.1197 34.7358 1.6148 Median 0.2043 0.1613 0.1212 35.3521 1.6140 Perc. 25% 0.1932 0.1284 0.1126 32.9628 1.6135 Perc. 75% 0.2410 0.1954 0.1243 36.2711 1.6157 Whisker low 0.1863 0.0994 0.0999 31.0943 1.6128 Whisker up 0.2769 0.9776 0.1439 38.4544 1.6224 HV normalized
ASEMOO GeDEA-II GeDEAII-K ParEGO RespSurf
Mean value 0.9942 0.9967 0.9981 0.2274 0.9616 Median 0.9953 0.9979 0.9980 0.2268 0.9622 Perc. 25% 0.9945 0.9972 0.9979 0.2196 0.9617 Perc. 75% 0.9955 0.9986 0.9983 0.2341 0.9623 Whisker low 0.9888 0.9781 0.9973 0.2036 0.9578 Whisker up 0.9955 0.9992 0.9986 0.2496 0.9624
Table 7.4: Test function ZDT1: box-plot statistics of D-metric and normalized Hyper- Volume (with reference point at (1, 4)).
fixed parameter value= 0.5, while in the other one it takes value 0.45, as the leg- end reports. Dealing with data in the ZDT4 box-plot and table, they refer only to the best curve. This comparison it is done because the best curve outperforms the true Pareto front, due to fitting error. These are not due to the fitting data set, but likely due to the fact that only one factor defines the surface and so it cannot reproduce very well the true Pareto front behaviour. On the other hand, the second response surface curve shows how much far it does place with such a little variation of the fixed parameter value.
Before the analysis of box-plot data, the set of points performing better than the true Pareto front has been handled. They were modified to realize a negative contribution to the Hyper-Volume. Dealing with the D-metric, this is not nec- essary since it always measures the discrepancy of points from the true Pareto front. Finally, observing the box-plot it is possible to see that both D-metric and HV are anyway very good. Notice that the reference point is placed far from the true Pareto front to display results also from the ParEGO algorithm. In fact also ASEMOO, whose results lay over the second response surface ones, has an HV measure almost equal to 1, as the best algorithms3.
ZDT6 In this last two-objective test function, response surface is able to retrieve just a little part of the Pareto front, again due to fitting problem, as reported in
3It is possible also to notice that the Pareto front plot B.5 is reported in a logarithmic y-scale to display all the results in a proper way. Otherwise, it would not be possible to observe clearly that response surface performs better than the true Pareto front. However, even if there are no other plots in logarithmic scale and they cannot highlight this particular behaviour, data show that this behaviour does not happen in other test functions.
figure B.7. However, it obtains results that are exactly on the true Pareto front. Though on the other hand, due to the little front build, the performances of D-metric and normalized HV are quite low. Even if HV box-plot marks response surface as the worst model, the D-metric shows that it does not perform so bad, again because of the accurate result of the ending part of the true Pareto front.