Composición del Consejo de Administración

The mixture surrogate model algorithm SO-M [98] has been further devel- oped with respect to two major aspects, namely the method of choosing the next sample site, and the computation of the model characteristics. Comput- ing the model characteristics is done by leave-one-out cross-validation in the algorithm described in Chapter 2. This approach becomes a computational burden as the number of sample sites and the problem dimension increase because the models have to be fit for every sample site that is left out. The complexity of computing the model parameters depends on the problem dimension and the number n of already sampled points. Therefore, the leave-one-out cross-validation is applied only while the number of sample sites is less than 50. Thereafter, the ˜k-fold cross-validation is applied, where ˜

evaluated.

In the ˜k-fold cross-validation, iteratively a subset of ˜k sample points is taken out of the total setS of already sampled points. This subset is called the validation set. The remaining n− ˜k points constitute the training set. Using the points in the training set, all surrogate models are fit to the data. Each surrogate model is used to re-predict the objective function values of the sample points in the validation set. This is done for each subset, and based on the re-predicted and the true function values, correlation coefficients, maximum absolute errors, median absolute deviations, and root mean squared errors are computed for each model. These values are then transformed into basic probability assignments as described in Section 2.2, and DST is used to determine the weights of the individual models in the mixture in equation (2.2).

The second change made in the SO-M algorithm was to replace the method for finding the next sample site. Since studies with RBF models showed that a stochastic sampling strategy may be more successful than optimizing an auxiliary function [119], a randomized approach is used. The SO-M algorithm uses either the minimum of the response surface or a target value strategy. In the target value strategy, the search is either global (the best point is searched outside a densely sampled area of the variable domain), or local (the target value strategy is applied within the densely sampled area). Thus, SO-M is able to escape from local optima and explore other promising regions of the variable domain. In every iteration several target values are used, and therefore several auxiliary optimization problems have to be solved. For low-dimensional problems this is computa- tionally inexpensive, but the computational complexity increases with the problem dimension. Therefore, a randomized sampling procedure similar to the candidate point approach by Regis and Shoemaker [119] has been used. In the randomized sampling procedure candidate points (denoted by χ_j, j = 1, . . . , t) for the next sample site are generated as follows. One group of candidates consists of points that are uniformly selected from the whole variable domain Ω. The points in the second group are generated by perturb- ing the best point found so far, i.e. x_best = argmin_ι=1,...,nf (x_ι). In contrast to [119] where all variables are perturbed with the same perturbation range, every variable is perturbed with probability

P = 

max{0.1, 5/k} if k > 5

CHAPTER 3. SO-M-C AND SO-M-S 65

and three perturbation ranges have been used to obtain large, medium, and small perturbations. Thus, when generating the candidate points in the second group, randomly chosen variable values of x_best are perturbed by randomly adding or subtracting small, medium, or large perturbations. This allows the generation of a broader range of candidate points for which the magnitude of the perturbations of all variables may be different. Thus, a larger diversity of points in the vicinity of x_best is generated as compared to the approach in [119].

Two criteria are then used to find the “best” candidate point [119]. The first criterion is determined based on the distance of every candidate point to the set of already sampled pointsS (“distance criterion”). The second criterion is based on the objective function value predicted by the mixture surrogate model (“response surface criterion”). A score is computed for each candidate point as a weighted sum of both criteria. The candidate point with the best score becomes the point for doing the next expensive function evaluation. A global search can be achieved by giving candidate points that are in relatively unexplored regions of the variable domain preference, i.e. by assigning a high weight to the distance criterion. On the other hand, once a promising point has been found, its vicinity should be explored more thoroughly. A local search can be achieved by giving a larger weight to the response surface criterion because the response surface is likely to predict lower objective function values for points in the vicinity of xbest.

By repeatedly cycling through a weight pattern for the criteria, a repeated transition from global to local search is achieved [119]. The algorithm starts by assigning a large weight to the distance criterion and a low weight to the response surface criterion. The weight for the distance criterion is iteratively decreased, and the weight for the response surface criterion is increased. After the distance criterion weight has reached its minimum (and the weight for the response surface criterion has reached its maximum), it is re-initialized to the maximal value (minimal value). The advantage of this randomized sampling strategy is that no subproblem has to be optimized to find the next sample site, and savings in computation times are possible. The algorithm is implemented such that it satisfies the convergence conditions stated in [119] and convergence follows from Theorem 1 in [119]. Figure 3.1 illustrates the two criteria and the weighted scoring function for a one-dimensional problem (note that illustrated are the values scaled to [0,1]). The scoring function (solid green line) is not unimodal, and finding the

global minimum would require a global optimization algorithm. Numerical experiments on a set of test problems in Chapter 5 showed, however, that trying to find the global minimum of the scoring function and using the corresponding point as the next sample site does not lead to better results than the candidate point approach.

−10 −8 −6 −4 −2 0 2 4 6 8 10 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 x Score Response surface Distance criterion Score function

Figure 3.1: A one-dimensional example showing the response surface and distance criterion together with the weighted sum (score function) of both

criteria.

A third alteration of the SO-M algorithm in Chapter 2 is that the surrogate models contributing to the mixture are fixed at the beginning of the algo- rithm, and during the optimization phase only their weights are adjusted, as opposed to the original implementation where the surrogate models contributing to the mixture could change throughout the optimization phase. By this approach it is possible to better examine the influence of single models on the mixture. The specific steps of the stochastic mixture surrogate model algorithm SO-M-c (Surrogate Optimization - Mixture -

CHAPTER 3. SO-M-C AND SO-M-S 67

Algorithm 4 SO-M-c: Mixture Surrogate Model Algorithm With Candidate Point Sampling Strategy

1. Create an initial experimental design using a symmetric Latin hyper- cube sampling strategy. Evaluate the computationally expensive objec- tive function at the generated points.

2. Apply cross-validation to compute the characteristics of the desired sur- rogate models in the mixture:

• If n ≤ 50, use the leave-one-out cross-validation.

• If n > 50, use the ˜k-fold cross-validation, where ˜k increases with the number of function evaluations (˜k = 10 for 50 < n ≤ 100, ˜

k = 20 for 100 < n≤ 150, ˜k = 30 for 150 < n ≤ 200, etc.) 3. Use Dempster-Shafer Theory to determine the weights w_r of the chosen

models in the mixture in equation (2.2).

4. Compute the parameters for each model contributing to the mixture. 5. Generate candidate pointsχ_j, j = 1, . . . , t, by uniformly selecting points

from the variable domain and by perturbing x_best as follows. With δ = maxi=1,...,k{xui − xli}, the perturbations are computed as gδΥ, where

the standard deviation g ∈ {0.1, 0.01, 0.001} is chosen randomly, and Υ ∼ N (0, 1) is a random variable drawn from the standard normal distribution. Every variable of x_best is perturbed by adding gδΥ with probability P as defined in equation (3.4) to its value.

6. Compute the distance

Δ(χ_j) = min

ι=1,...,nχj− xι2, (3.5)

where  · ₂ is the Euclidean norm in Rk_{, and x}

ι ∈ S. Scale the values

to the interval [0, 1]:

V_D(χ_j) =

_{Δmax−Δ(χ}

j)

Δmax−Δmin if Δmin = Δmax

1 otherwise , (3.6)

where V_D(χ_j) is the scaled distance value for candidate χ_j, Δ_max = maxj=1,...,t{Δ(χj)}, and Δmin = minj=1,...,t{Δ(χj)}. Eliminate candi-

7. Use the mixture model in equation (2.2) to predict the objective function values s_mix(χ_j) of the candidate points, and scale these values to the interval [0, 1]:

VR(χj) =

_smix(χ

j)−smin

smax−smin if smin= smax

1 otherwise , (3.7)

where V_R(χ_j) is the scaled predicted objective function value for candi- date χ_j, s_max = max_j=1,...,t{s_mix(χ_j)}, and s_min= min_j=1,...,t{s_mix(χ_j)}. 8. Compute the weighted scores of the candidate points

V (χ_j) = ω_RV_R(χ_j) + ω_DV_D(χ_j), j = 1, . . . , t, (3.8) where ω_R+ ω_D = 1, and ω_R ≥ 0 is the weight for the response surface criterion, and ω_D ≥ 0 is the weight for the distance criterion. Set ω_D = 1 in the first iteration and decreases the value for 0.1 in every following iteration until ω_D = 0. Reinitialize the value to ω_D = 1, and decrease the value anew, etc. Choose the candidate point with the best score (the smallest value for V ) as next sample site.

9. Do the costly function evaluation at the chosen point.

10. If the maximum number of allowed function evaluations has not been reached, go to Step 2. Otherwise, return the best point found.