condition b is chosen as a fixed, known value and thus the parameters of the model are the entries of the matrix Λ. Then, a P Rn2 is the parameter vector consisting of the matrix entries of Λ. ρpaq is defined as a multivariate Gaussian distribution a „ N p¯a, Σq with mean vector ¯a and covariance matrix Σ “ σ2I where I is the n2ˆ n2 the identity matrix. Then, the density function is defined to be
ρpaq “ 1
ap2πq4|Σ|exp ˆ´1
2 pa ´ ¯aqJΣ´1pa ´ ¯aq
˙ .
Figure 25: Visual summary of approach 2. The red coloring indicates the first density that is defined to begin the approach. KDE refers to the kernel density estimation of Y with a density ˜ηpyq.
When, the sample the A “ ta1, a2, . . . aNu is randomly drawn from the density ρpaq, we restrict to parameters for which the data is uniquely produced by that parameter (for a P A, F paq “ y and there is no other b P A such that F pbq “ y). This can easily be achieved in practice by selecting a mean parameter ¯a with real, distinct eigenvalues and taking σ to be small. Even if σ is small, aj drawn from a Gaussian distribution might fail to have real, distinct eigenvalues, in theory. Only aj with real, distinct eigenvalues are included in the sample. The solution map F is then used to find the corresponding data F pajq “ yj, which yields the sample Y “ ty1, y2, . . . yNu. Due to the restriction on the sampling, it follows that F : A Ñ Y is a bijection as desired. The value N is chosen to be large, to gain as much information about the structure of the data density as possible.
We aim to diminish the error in the posterior sample, M , resulting from the choice of
how the data are represented and which likelihood function is used. In order to achieve this objective, we seek a density function which is as close as possible to the underlying distribution that generated the sample Y . For the finite sample of observed data, density estimation constructs an estimate of the underlying probability density function everywhere, including where no data are observed. Histograms are of the most simple density estimators;
however, they are not smooth and the choice of bin width leads to a constant struggle between bias and variance [72]. Kernel density estimation is a more sophisticated alternative. In this approach, a kernel function is centered at each data point, so that a discrete data point is smoothed over the region surrounding it. The kernels are then summed, resulting in single function that is the estimated density function. Background on kernel density estimation is provided in Appendix B.
In MATLAB, the built in function ‘ksdensity’ performs a kernel density estimation for finite sets of univariate and bivariate data. In the setting of this work, the data have dimension nm, which will be larger than 2 in all of the following examples. MATLAB code for multivariate kernel density estimation based on the papers [54, 55] has been made freely available by the authors. One desirable feature of this toolbox is that the bandwidth does not need to be prescribed and is optimally determined within the program. It can also handle large data sets and performs a pre-clustering in order to do so. This code, with minor modifications, was used for all density estimates presented in this work. Figure 26shows a 4-dimensional kernel density estimate obtained using this code. In this figure, marginals of the data are represented by blue histograms and marginals of the density estimate are given by the red curves. The small nest of black curves depict marginals of the individual kernels that were summed for the density estimate.
Using the same data set as Figure26, two-dimensional projections of the data and density estimate are given in Figure27. In part (a), projections of the discrete data set are depicted and in part (b), two-dimensional projections of the density estimate are shown, with the third dimension represented by a color shading. Both of these figures indicate that the density estimate produces an accurate representation of the data.
Returning to the implementation of Approach 2, an approximate density of sample Y is constructed with multivariate kernel density estimation, using the freely available MATLAB
0 5 10
Figure 26: Example kernel density estimate for 4-dimensional data. Marginal histograms of the data are shown in blue, marginals of the density estimate are given by the red curves, and marginals of the kernels surrounding the data points are given by the black curves.
5 10
Figure 27: Example kernel density estimate for 4-dimensional data. Two dimensional pro-jections of the discrete data set are pictured in (a) and propro-jections of the kernel density estimate are given in (b).
code from [54] and is denoted by ˜ηpyq. Next, the Metropolis-Hastings algorithm is performed and Jeffreys prior, the uniform prior, and the Jacobian prior are employed in order to obtain MJ ef f, MU nif, and MJ ac, respectively.
In the implementation of the Metropolis-Hastings algorithm, the density estimate ˜ηpyq
will be used in the computation of the likelihood in the same manner that the exact data density ηpyq was used in Approach 1, namely Lp˜η|aq “ ˜ηpF paqq. The proposal density and the numerical estimation of the Jacobian will be the same as presented in section 4.4.1.2.
The accuracy of the kernel density estimate reduces the error in the posterior estimate due to incomplete knowledge of the data and the choice of likelihood, and therefore the effect of the prior density will be revealed.
4.4.1.6 Approach 2 examples In this section, several examples of Approach 2 will be