In contrast toχ2-minimization techniques, Bayesian analyses aim to construct the poste-
rior probability of the model parameters given the data, or p(θ|D). To do this, we make
use of Bayes’ theorem, which relates the posterior probability to the likelihood p(θ|D) =p(D|θ)p(θ)
p(D) =L
p(θ)
p(D). (2.13)
It is often more useful to rewrite Bayes’ theorem in the context of a particular modelM
which is a function of the parameter setθ. In this case, Eq. 2.13 becomes p(θ|D) =LR p(θ|M)
L(D|M(θ))p(θ)dθ . (2.14)
The probabilityp(θ|M)in Eq. 2.14 is known as theprior. The prior describes where
we believe the true value of the parameters lie and is generally informed by the data and current or past experiments. While the inclusion of a prior is controversial to many frequentists, well-motivated priors, e.g., a physical prior requiring galaxy mass to be pos- itive, ensure that we are using prior information to the best of our ability before gathering and analyzing the data. While there exists a wide range of prior functions, there are a few which are common in the SN Ia literature: uniform, Gaussian, and Jeffreys. A uniform (flat) prior indicates that all points in parameter space are equally probable, but explicitly prohibits points in parameter space outside the prior range. The shape of Gaussian priors can be easily manipulated, and their long tails do not definitively exclude specific parts of parameter space. A Jeffreys prior goes as 1/θ, is uniform in log space, and is often
used for inference of scale parameters. The choice of priors primarily depends on how much information is available. Broader priors should be used in cases with little infor- mation; narrow priors, e.g., from a well-tested theory, are useful in cases with poor data. Ultimately, the choice of prior should not dominate the likelihood. Analyses where the prior is the dominant driver of the posterior indicate that the data cannot constrain the parameters of interest or that the prior was inappropriately chosen.
The denominator of Eq 2.14 is the product of the likelihood and prior integrated over all points in parameter space. This quantity is often referred to as themodel evidenceand is used to determine the “best” model in a set of competing models.
Just as the likelihood is sampled in theχ2-minimization technique, the joint posterior
probability, p(θ|D), is sampled in Bayesian inference. Rather than sample the full joint
posterior, however, we often sample the product of the likelihood and the prior as this is proportional to the posterior and does not require the integration over the full range of parameter space. Therefore, a likelihood is also essential for this method of parameter inference.
2.4.1.1 Bayesian Hierarchical Models
Linear regression with uncertainties in both the independent and dependent variables, as is the case with the SN Ia cosmology problem, is nontrivial in the classical Bayes formalism. The Bayesian Hierarchical Model (BHM) framework was introduced in Gull (1989) to address this issue. Gull proposes a two-part solution:
1. Hidden variables, which represent the latent or “true” values of measured quantities are introduced. These are treated as nuisance variables and ultimately marginalized over.
2. Informative priors are imposed on the hyperparameters, parameters describing the latent variables. These priors are particularly important for hyperparameters repre- senting the locations of the latent variables, e.g., the mean of a distribution.
Gull (1989) asserts that this hierarchical or “sub-model” structure recovers unbiased esti- mates of the parameters, particularly of the slope parameters, as long as informative priors are included. Here, bias in an estimated parameter refers to a systematic deviation from the true value of the parameter, i.e.,
bias≡Dθbest fit−θtrue E
and can only be estimated over multiple realizations of the data.
Figure 2.1 displays a sample BHM for a simple toy regression problem with errors in the independent (xi) and dependent (yi) variables (March et al., 2011). In Figure 2.1, solid lines indicate probabilistic connections and dashed lines indicate deterministic connec- tions. Parameters to be constrained are circled in red, latent variables are circled in blue, and data are circled in green. As shown in Figure 2.1, there are two types of parameters to be constrained: the conventional set of model parameters (θ) and the hyperparameter
describing the width of the latentx distribution (Σx). To achieve unbiased estimates of θ, informative priors must be imposed on the hyperparameters. The classic Bayesian ap-
proach would not include the hyperparameter Σx which describes the distribution of the latentxi.
2.4.1.2 Bayesian Inference with SNe Ia
Recently, Bayesian inference has become a more popular technique in SN Ia cosmology analyses due to its flexibility and computational efficiency. The BHM framework easily incorporates a variety of SN Ia standardization models and can be used to explore model nuances and build sophisticated model networks.
Figure 2.2 features two example hierarchical frameworks designed for cosmological parameter inference using SNe Ia. The top panel of Figure 2.2 displays the BHM network presented in March et al. (2011), the first application of BHM to the SN Ia cosmology problem. The bottom panel features a more recent and complex BHM presented in Rubin et al. (2015). In both networks shown in Figure 2.2, dashed lines indicate deterministic relations and solid lines indicate probabilistic relations. Both models include cosmolog- ical parameters and the SALT2 SN Ia standardization coefficients α and β. They also
include hyperparameters describing the position and scale of the latent light-curve color and stretch distributions. The Rubin et al. (2015) model builds on that of March et al. (2011) by including other parameters such as host-galaxy standardization coefficients and parameters describing systematic uncertainties and sample limiting magnitudes. For fur-
Figure 2.1: Sample BHM reproduced from March et al. (2011). Solid lines indicate probabilistic connections; dashed lines indicate deterministic connections. Parameters to be constrained are circled in red, latent variables are circled in blue, and the data are circled in green. The classic Bayesian model would not include theΣx hyperparameter describing the distribution of the latentxi.
ther examples of the diversity of BHM models applied to SN Ia data, see Mandel et al. (2009), Shariff et al. (2016), and Mandel et al. (2016).
In many SN Ia BHM analyses, deriving the analytic form of the likelihood involves change of variables, marginalization over latent variables and nuisance parameters, as- sumptions about variable covariances, etc. This leads to analytic prescriptions of the likelihood that are rather complex and may be incomplete. For example, after marginaliz- ing over latent variables, nuisance parameters, and SN Ia redshift uncertainties, the March et al. (2011) likelihood is expressed by
p(d|θ) = Z dlogRcdlogRx|2πΣC|−1/2|2πΣP|−1/2|2πΣ0|−1/2|2πK|1/2× exp −1 2 X T 0 ΣC−1X0−∆ T Σ−A1∆−k0TK−1k0+bTmΣ−01bm . (2.16)
Definitions of the parameters used in the likelihood are described in Appendix C of March et al. (2011). We do not define them here as we include the likelihood merely as an illustrative example of SN Ia BHM likelihood complexity.
Rubin et al. (2015) employ several variations of their BHM for cosmological parame- ter inference, using the Union2.1 compilation of 580 SNe Ia assembled by the Supernova Cosmology Project (Suzuki et al., 2012). When comparing their BHM posteriors to the corresponding best-fit results obtained using the traditional χ2-minimization approach
and the same SN Ia standardization model, they find their 1-D marginalized posteriors give roughly the same 1σ uncertainty region forΩm.