The discussion moves next to the examination of the impact of the metamodel accuracy. This is established by considering additionally the results obtained by using the exact stochastic ground motion model (i.e., not relying on metamodel predictions), which represents the measure for evaluating the actual hazard com- patibility of the identified ground motion model. The details for the study are the ones used in the previous section. Results for seismicity scenario M =6-R=20km are presented in Figures 4.5 and 4.6. Figure 4.5 shows the Pareto fronts identified by using the metamodels with the three different number of support points. Cases with or without the metamodel error in the estimation of objective function F1are
separately shown. This leads to two different Pareto sets, one without error and one with error, and for each two different fronts are reported, one corresponding to metamodel predictions and one to the use of the exact stochastic ground mo- tion model. Then Figure 4.6 shows the spectral plot comparisons for the solution (among the Pareto set identified in each case) corresponding to the minimum of F1. The period range used in Figure 4.6 corresponds to the target structural peri-
ods Ts. In all cases, the exhaustive search is implemented with the same candidate
then shows Pareto front results for different seismicity scenario, M =7.8-R=30km. Note that for seismicity scenario M =7-R=40km (another case discussed in the pre- vious section), results are of limited interest since the unmodified ground motion model provides a good compatibility to target IMs.
0 0.2 0.4 0.6 0.8 1 0 0.5 1 1.5 2 2.5 3 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
(A) Metamodel with 1500 support points (B) Metamodel with 3000 support points (C) Metamodel with 4500 support points Metamodel
Actual model Black
Grey
Without metamodel error With metamodel error
1
F F1 F1
2
F
Figure 4.5: Pareto fronts identified using metamodels with (A) 1500, (B) 3000, or (C) 4500 support points and comparison to predictions by exact stochastic ground motion model. Case presented corresponds to seismicity scenario M =6-
R=20km. 0.5 1 1.5 2 0 0.05 0.1 0.15 0.2 0.5 1 1.5 2 0.5 1 1.5 2 P S A (g) Target Metamodel predictions Actual model predictions Black
Grey
Without metamodel error With metamodel error
(A) Metamodel with 1500 support points (B) Metamodel with 3000 support points (C) Metamodel with 4500 support points
T (s)s T (s)s T (s)s
Figure 4.6: Spectral plots for the solutions corresponding to minimum of F1 in the Pareto fronts identified in Figure 4.5.
The results show that for the higher accuracy metamodel (4500 support points) good agreement is established between the metamodel predictions and the actual model predictions along the Pareto front, whereas the inclusion of the metamodel error has only a small effect on the identified Pareto front. This is observed in both Pareto fronts (Figures 4.5 and 4.7) as well as in the corresponding spectral
0 0.05 0.1 0.15 0.2 0.25 0.3 0 0.5 1 1.5 2 2.5 3 0 0.05 0.1 0.15 0.2 0.25 0.3 0 0.05 0.1 0.15 0.2 0.25 0.3 Metamodel Actual model Black Grey
Without metamodel error With metamodel error
(A) Metamodel with 1500 support points (B) Metamodel with 3000 support points (C) Metamodel with 4500 support points
2
F
1
F F1 F1
Figure 4.7: Pareto fronts identified using metamodels with (A) 1500 (B) 3000 or (C) 4500 support points and comparison to predictions by exact stochas- tic ground motion model. Case presented corresponds to seismicity scenario
M =7.8-R=30km.
plots (Figure 4.6). Different trends are observed, though, for the lower accuracy metamodel (1500 support points). For lower F1 values, and therefore large F2 val-
ues, the Pareto optimal solutions identified when metamodel error is not included in the problem formulation lead to erroneously modified ground motion models. Based on the metamodel predictions, these models provide a very good match to the target IMs (small predicted F1 value), but when response is evaluated with the
exact model larger differences are observed from the target IMs. This is particu- larly evident in the spectral plots shown in Figure 4.6. Evidently the respective solutions identified correspond to parameters θ for which the metamodel accuracy is low. The moderate accuracy metamodel (3000 support points) falls in between the two aforementioned cases, with characteristics that resemble more closely the ones for the high accuracy metamodel.
Note that the seismicity case examined here is ideal for exploring vulnerabilities in the optimisation associated with lower metamodel accuracy, as it corresponds to a case at the boundary of the scenarios used to define domain X. Therefore, for larger values of F2, the corresponding parameters θ are expected to be close
to the boundary of X, where metamodel accuracy is lower. However, even for this challenging case, metamodels with higher accuracy (3000 or 4500 support points) face small challenges, whereas the inclusion of the metamodel error for the
lower accuracy metamodel (1500 support points) greatly improves the robustness of the optimisation, leading to identified solutions with good agreement between metamodel and actual model. Figure 4.8 sheds further light into this topic. It focuses on the case examined in part (A) of Figure 4.6 but besides the mean metamodel predictions it includes the predictions that are 1.5 standard deviations (σ) from the mean based on the estimated error variance. The solution identified when metamodel error is not included in the evaluation of F1 is associated with
a larger anticipated error (part (A) of Figure 4.8). Note that the actual model is actually even further away than the plotted 1.5σ. When the metamodel error is included in the evaluation of F1, such θ values with large associated σ are
avoided, since the large σ contributes to larger values for the objective function. This ultimately contributes to identification of solutions with greater robustness, i.e., better agreement between metamodel and actual model (part (B) of Figure 4.8). 0.5 1 1.5 2 0 0.05 0.1 0.15 0.2 0.5 1 1.5 0 0.05 0.1 0.15 0.2
(A) Optimization without error term (B) Optimization with error term
P S A (g) P S A (g) Ts (s) 2 Ts (s) Mean ± 1.5 standard deviation Target Mean predictions Actual model predictions Metamodel
Figure 4.8: Spectral plots for the solutions corresponding to minimum of F1in the Pareto fronts identified in Figure 4.5 for the metamodel with 1500 support points. For the metamodel predictions the mean predictions and the predictions
within 1.5 standard deviations from the mean are shown.
Overall, the above discussion shows that metamodels with higher accuracy (coeffi- cient of determination 98%) can be considered as a good surrogate for the proposed optimisation, whereas the inclusion of the prediction error greatly improves the robustness of this optimisation, avoiding identification of erroneous solutions, even
when metamodels with lower accuracy are adopted. Consideration of this error is not necessary, though, when higher accuracy metamodels are used.