HS 5 EVACUACIÓN DE AGUAS

6.2. Time to assess new points for engine data sets

Like in the previous two chapters the results related to the engine data sets are presented in a separate section. In the remains of this chapter the assessment time for the different engine test cases can be found.

6.2.1. 2D - engine data

The time required to assess new test points in the two-dimensional hypothetical data case shows similar results to the engine data case, as can be seen in Figure 6.5. Even the individual amount of assessment time is almost the same in both cases and are independent of the selected output. The main difference between the two result plots is that the PEV method does seem to need a little more time to assess new points with an increase in number of training data.

CH PEV SVM SVM−LOO CH PEV SVM SVM−LOO

0 0.05 0.1

Number of training points (incr. with arrow), method and boundary (BEFF, SPMI)

Calculation time [s]

Figure 6.5.: Time to test new points for a 2D engine training data set

6.2.2. 3D - engine data

In Figure 6.6 the results for the three-dimensional engine data set can be found. It again is similar to the same-dimensional hypothetical case. Compared to the two-dimensional engine case the assessment time about doubles for all methods and number of training points. The results do not significantly differ for the BEFF and the SPMI case.

6.2.3. 4D - engine data

As for the four-dimensional hypothetical case, the time to assess new points in the engine case takes about double of that of the three-dimensional case, shown in Figure 6.7. All methods show

CH PEV SVM SVM−LOO CH PEV SVM SVM−LOO 0

0.05 0.1

Number of training points (incr. with arrow), method and boundary (BEFF, SPMI)

Calculation time [s]

Figure 6.6.: Time to test new points for a 3D engine training data set

an increase in assess time for a larger number of training points, where the gradient of PEV is slowest. No obvious difference due to the test case or boundary shape can be detected.

CH PEV SVM SVM−LOO CH PEV SVM SVM−LOO

0 0.05 0.1

Number of training points (incr. with arrow), method and boundary (BEFF, SPMI)

Calculation time [s]

6.2. Time to assess new points for engine data sets

6.2.4. 7D - engine data

The seven-dimensional results can be found in Figure 6.8. The results are shown for training data sets up to 2500 points. The CH method clearly needs the most time to assess new points, but it is significantly less time than for the similar hypothetical case up to2500 training points. The PEV and both SVM methods need approximately the same amount of time as in the hypothetical case. PEV is again the fastest method and does not seem to be affected by the number of training points. The SVM and SVM-LOO method show an increased assessment time with an increasing number of training points, where the SVM-LOO has a steeper gradient.

CH PEV SVM SVM−LOO 0 5 10 15 20 25

Number of training points (incr. with arrow) and method

Calculation time [s]

(a) Assessment time

CH PEV SVM SVM−LOO 0 1 2 3 4 5

Number of training points (incr. with arrow) and method

Calculation time [s]

(b) Assessment time zoom

6.3. Chapter conclusions

As in the previous two chapters the research question is initially discussed by the corresponding subquestions, following the same order as before. In the last section a final answer will be given to the third and last research question of the study.

RQ3a What is the influence of the problem dimension on the time to assess new test points?

The problem dimension has an effect on the time required to allocate new test points for all methods. Higher dimensions cause the time to assign new points to increase, but up to four dimensions the calculations are still quite fast and within 0.2 seconds. The CH method is the fastest method up to four dimensions, but for the seven-dimensional case it needs most time. The PEV methods follows the CH method closely up to four dimensions where assessment time is concerned. In seven dimensions the PEV method is the fastest and the only method that can do all its calculations with in1 second.

RQ3b What is the influence of the number of training points on the time to assess new test points?

The number of training points have an influence on the time of assessing new points. The more training points are used the longer the allocation time. But the effect is not so strong as for the time required to train the boundary.

As expected the number of training points do not largely affect the assessment time for the CH and PEV methods. The two methods train the boundary with the required data, store the results describing the boundary and assessing new points can be achieved by easy matrix calculations. For the SVM-based methods it is expected that the number of training data influences the as- sessment timing, since the kernel matrix, used for the kernel trick, needs to be recalculated for every single test point.

RQ3c What is the influence of the true boundary shape on the time to assess new test points?

The shape of the boundary does not influence the assessment time of new points.

RQ3Which design space description method is fastest to assess new test points?

Analysing the three previous questions it can be said that up to four dimensions the CH-method is fastest, followed closely by the PEV method. For the seven-dimensional cases the PEV method is much faster than all other methods.

Comparing only the two SVM-based methods it can be seen that up to four dimensions the SVM-LOO method was faster, but in seven dimensions the SVM method was faster.