FUENTES DE INFORMACIÓN

6.4.1 Introduction

In Sections 6.3.2 and 6.3.3 we built emulators of the radiation transport model to provide an efficient method for performing further analysis on the model. We now use the emulators to perform calibration, finding what input settings are consistent with a given model response. Observational data is unavailable, so as an alternative we choose one run from the training set of 100 to represent such an observation. The 22nd run is chosen by the domain expert as an interesting setting of parameters. The inputs and outputs for this run are given in Table 6.2.

Inputs Outputs

Native Transformed r₁ = 7.6113 γ₁ = 4.132718e-07 -14.6991603 ρ₁ = 18.6040 γ₂ = 2.308216e-04 -8.3738654

ρ₂ = 7.2778 γ₃ = 2.134913e-04 -8.4519145 z₁ = 4.3119 γ₄ = 1.652884e-03 -6.4052336 z₂ = 6.4739 1/k_{ef f} = 2.039376e+00 0.4903461

Table 6.2: Inputs and outputs for run index 22, chosen to be the target output.

To do this we use a Markov chain Monte Carlo (MCMC) method with the emulators to learn which other sets of inputs are consistent with the output from run index 22. MCMC and the method we employ is discussed further in Section 6.4.2 along with the results from this applied to the radiation transport model.

6.4.2 MCMC applied to radiation transport model

If we are interested in inferences from a posterior distribution but cannot analyti-cally derive the distribution then we can use Markov chain Monte Carlo methods.

We generate a Markov chain and the stationary distribution of the chain is equal to the desired posterior distribution.

One way of constructing such a chain is by using the the Metropolis-Hastings algorithm: let x_t be the state of the chain a time t.

1. Choose a proposal distribution, q(x_c|x_t).

2. Generate a candidate value x_c from the proposal distribution.

3. Calculate the likelihood ratio and the ratio of the proposal density between the current sample, x_t and the candidate sample, x_c,

4. Draw α from U(0,1).

5. If ^{f (x}_{f (x}^c)q(xc|xt)

0)q(xc|xt) > α; then accept the candidate value and x_t+1 = x_c. else; reject the candidate value and x_t+1 = x_t.

To learn which other input configurations are consistent with our target output we apply MCMC methods using emulators. From the results of the investigation into the comparison of multivariate and univariate emulators, with and without derivatives, we choose to build univariate emulators for all outputs and include derivative information only in the emulator of output 5. We remove the target output from the training data and build an emulator, for each output, with the remaining 99 runs in the data set.

We apply the Metropolis-Hastings algorithm as outlined above. We take the following, approximately mid-range, points for the initial values of the inputs:

x₀ = (10 11 11 2.5 5) and generate the first candidate value: x_c = x₀+ U (0, 1).

We use the following likelihood function:

exp

where m^∗∗_i is the posterior mean of the emulator, t_i is the target output and sd

= 0.1, which represents observational error and is scaled to the measurement. A uniform prior over the original parameter range results in a posterior distribution conforming to the likelihood.

We generated 100,000 samples from the posterior distribution. Although the radiation transport model is relatively quick to run in comparison with other complex models, performing 100,000 runs would still take an order of days. By using the emulator instead, the MCMC analysis was complete in approximately an hour.

The results are summarised in Figure 6.13, which shows two-dimensional marginal projections of the 5 dimensional space for the last 5000 iterations. Along the diagonal are histograms showing the density for each input parameter, where the bottom left panel shows input 1 through to the top right panel which shows

input 5. The upper panels show contours and the lower panels show two dimen-sional scatter plots between the inputs, where the black points are from earlier in the chain and as the chain progresses move to the light colours. The blue star shows input 22, the observation we are comparing to. Firstly, we can see that for each input the chain covers the true input configuration of the observation which is encouraging. It appears that to achieve similar output to the observation, it is plausible for input 5 to take most values across its input range. In contrast, input 3 appears to be more tightly constrained. In addition to this we can see that there exists a tradeoff between the first and third input parameter, which represents inner radius and outer density respectively.

Input 1 Input 2 Input 3 Input 4 Input 5

Input 1Input 2Input 3Input 4Input 5

Figure 6.13: Two-dimensional plots from the MCMC results.

6.5 Conclusions

In this chapter we have investigated the use of derivative information in both univariate and multivariate emulation, through the application of the radiation

transport model. We have not seen here any strong evidence in support of building a multivariate emulator, with or without derivatives, over independent emulators for each output. The impact of derivative information has been inconsistent, with the derivatives proving more informative for some outputs than others, in both the univariate and multivariate environments.

Despite investigating the use of derivative information in multivariate emu-lation we still chose to use univariate emulators in the calibration of the radia-tion transport model in Secradia-tion 6.4. This is because the multivariate emulator performs similarly to the independent emulators, and as noted above, therefore provides no strong reason to choose an emulator other than univariate emulators for the calibration. Given all the data available, derivatives are not required to provide accurate emulation of outputs 1 to 4 and therefore are not used in the calibration study. We have seen here and in Chapter 4 though, that derivative information does improve the emulation of output 5 and therefore an emulator built with function output 5 and the corresponding derivatives is selected.

In summary, suppose we wish to build an emulator of the 5 outputs of the radiation transport model and have only a finite amount of computational time in which to run the simulator or adjoint. We describe this time in terms of to-tal computational units, T . In this chapter we have attempted to answer the question: should we build 5 independent emulators or one multi-output emulator and, given the type of emulator, would it be more efficient to include derivative information? We have not seen any strong evidence to support choosing the multivariate option over univariate and so for this model, the answer to the first part of the question is straightforward. Whether it is more efficient to include derivatives in the independent emulators however, is much more difficult to an-swer. From Figures 6.10 and 6.11 we see that for outputs 1 and 2, derivative information has little effect and we could choose either to run the adjoint at ^T₂ input sites, or the simulator at T input sites. Emulators of outputs 3 and 4, however, clearly do not benefit enough from the inclusion of derivatives to justify the computation expense of the adjoint. In Chapter 4 we see that emulators of very smooth models seem to benefit less from the inclusion of derivatives than emulators of models with a higher degree of variability. We do not adopt a formal measure to assess smoothness in the radiation transport model, but an indication can be drawn from the values of the smoothness parameters as estimated by MLE in the emulators. All inputs were scaled to be on the unit cube hence we can

combine the 5 θ values for each output in the independent emulators into one measure, for ease of comparison. We find that the mean values of θ for outputs 3 and 4, 0.285 and 0.294 respectively, are smaller than the corresponding estimates of θ for outputs 1, 2 and 5, 0.871, 2.507, 0.485 respectively. This implies that outputs 3 and 4 are smoother functions of the inputs than outputs 1, 2 and 5 and is consistent with the conclusions made in Chapter 4. For outputs 3 and 4, therefore, we should choose to perform T runs of the simulator. In contrast, the emulator of output 5 performs much better with derivatives included, despite the computational expense of the adjoint and the conclusion for this output would be to perform ^T₂ adjoint runs. The adjoint model, when run at a particular input configuration, produces all the partial derivatives of the 5 outputs with respect to the 5 inputs. It is not possible to ‘switch off’ the derivatives of outputs 3 and 4 and thus reduce the computational expense of running the adjoint. To achieve this a new adjoint model would have to be especially coded and therefore is very unlikely to be an efficient solution to our question. We are not restricted, however, to running either the adjoint at ^T₂ inputs sites or the simulator at T inputs sites. We suggest, therefore, that a mixture of adjoint and simulator runs is likely to be optimal in this case. This generates an interesting design problem and scope for further work: we would need to decide how many adjoint runs and how many simulator runs to perform, and also at which input configurations to obtain derivatives in addition to the model response.