In several real-world applications, a solution must be deployed in a time-varying, noisy and unseen environment for which it has not been optimized for. In this thesis, we make use of Bayesian networks for computing the posterior state distribution of such quantities, so to allow the search for solutions for which better performance on the targeted future environments is predicted. For stochastic MCDM, the idea is to achieve this goal by estimating the trajectory of random objective vectors over time.
The Kalman Filter [100] is the most widespread Bayesian network with countless successful applications, providing closed-form expressions for updating the state posterior distributions if the uncertainty can be modeled as a multivariate Gaussian and if the system dynamics are linear. As a result, exact and computational inexpensive one-step ahead inference of the state predictive distribution is available. The KF computes in closed-form the best possible estimation (in the mean-square sense) for the distribution of the hidden state vector. The main assumption for the
2.3. Learning and Prediction Under Uncertainty 47
KF is that of a linear Gauss-Markov state-space model, given as the following joint distribution factored representation of hidden states and measurements:
p(X, Y) = p(x1)p(y1|x1) H
Y
t=2
p(xt|xt−1)p(yt|xt) (factored joint distribution), (2.21)
x0 ∼ N (mx0, C0) (initial state distribution), (2.22)
xt|xt−1∼ N Amxt−1, Ct|t−1 (predictive distribution), (2.23)
yt|xt∼ N Mmxt, Rt|t−1 (measurement distribution), (2.24)
in which X = {x1, · · · , xH} ⊂ Rn is the set of unknown states, and Y = {y1, · · · , yH} ⊂ Rp is
the set of available measurements (which may have lower dimensionality, p < n). The matrices A and M encode the state linear dynamics and the measurement function, respectively. The uncertainty is encoded by one covariance matrix for the unknown states, Ct|t−1, and by another
covariance for the observations, Rt|t−1.
The equations for the KF come from the recursive application of the Bayes Theorem to the conditional densities p(xt|yt) and p(xt+1|xt, yt), corresponding to the updating of the posterior
given a measurement at time t, and the prediction of the next state using the known dynam- ical model, A. Both densities are Gaussian and can be analytically determined assuming the stochastic dynamical model in (2.21)–(2.24).
The KF is then capable of tracking state vectors evolving according to linear dynamics under Gaussian uncertainty. There are two steps required for recursively applying the KF for estimating a Gaussian random vector evolving on time: (a) a prediction step; and (b) a measurement correction step.
KF Prediction Step
This step makes use of the available dynamical model describing the trajectory of the hidden state vector over time to compute an estimation for the distribution of the hidden state vector in the current time step, given past estimations, as represented in Eq. (2.23). The dynamical model is assumed to be7:
xt = Atxt−1+ qt, qt∼ N (0, Qt) , (2.25)
where qt is a zero mean noise process with covariance Qt, which can be estimated from the
available historical data. The closed-form KF estimation is possible due to the mathematical tractability resulting from the linear algebra over Gaussian random vectors.
The computation of the predictive estimation of ˆxt|xt−1 from the prior distribution of ˆxt−1
is done over the parameters:
mxt|xt−1ˆ = Atmxt−1 (2.26)
Cxt|xˆ t−1 = AtCt−1A |
t + Qt, (2.27)
where Qt is the covariance of the additive noise process affecting the dynamics of xt. The
predictive distribution ˆxt|xt−1 thus expresses the KF prior belief of what the true hidden state
distribution xt might be.
7We assume no control input in our tracking application, i.e., the objective vectors are assumed to evolve
KF Measurement Correction Step
This step updates the estimated ˆx|xt−1 distribution obtained in the prediction step upon the
collection of a new sample yt resulting from measuring the hidden state xt. The measurement
distribution is denoted as yt|xt and results from the following expression:
yt = Mtxt+ vt, with vt∼ N (0, Rt), (2.28)
in which Mt is a measurement function model mapping the true state xt into a measurement
state ytand vtis a measurement noise process that is assumed as a zero mean Gaussian additive
noise process, with a noise covariance Rt that can also be estimated from available historical
data. The KF estimation for the current hidden state distribution given the measurement yt is
then represented as xt|yt and is also computed in closed-form over the parameters.
The closed-form equations arising from the KF measurement correction step make use of feedback from the residual between the observed measurement yt in Eq. (2.28) and the mea-
surement expected from the estimation ˆxt|xt−1 computed in the prediction step, i.e.,
˜
yt= yt− Mtxˆt|xt−1, (2.29)
where the residual ˜yt is also known as an innovation process containing novel statistical in-
formation that was not already known from the series of historical measurements y0, · · · yt−1.
Similarly, the residual measurement covariance matrix is expressed as St = MtCxtˆ |xt−1M
|
t + Rt. (2.30)
The so-called Kalman gain – represented as the matrix Kt – adjusts the importance of the
residual relative to the prior estimate ˆxt|xt−1 in the correction step of the true state distribution.
Intuitively, when the pairwise covariances in the measurement noise covariance matrix Rt are
high, less importance should be assigned to the observations and, therefore, the Kalman gain should be small. Conversely, when the estimated prior covariance Cxt|xˆ t−1 is high, the state
vector is expected to convey high variability and, hence, the Kalman gain should be high, to more strongly account for the observed measurements. In fact, the Kalman gain is the minimum mean-square error factor for correcting the prior estimation ˆxt|xt−1 while accounting
for the residuals resulting from the observed measurements yt and is expressed as
Kt= Cxt|xt−1ˆ M|TS−1t . (2.31)
The correction step is then completed by updating the parameters of the KF prior state esti- mation in light of the residual:
ˆ
xt = ˆxt|xt−1+ Kty˜t, (2.32)
Cxtˆ = (I − KtMt)Cxt|xt−1ˆ . (2.33)
Prediction in Metaheuristics Using KF
The first attempt to integrate KF estimation into metaheuristics was due to Stroud [199], who designed an extended KF to support an active learning heuristic for further reducing the
2.3. Learning and Prediction Under Uncertainty 49
uncertainty of the system by re-evaluating specific solutions. Particularly, an extended KF was designed to support heuristic decisions on “(...) when to generate a new individual, when to re-evaluate an existing individual, and which one to re-evaluate”.
Rossi et al. [175] applied the KF for tracking the optimum solution (for single-objective, uni- modal problems) and proposed several ad-hoc heuristics to incorporate the predictive knowledge in order to guide the evolutionary process toward promising regions. Three heuristic classes were assessed: (i ) incorporating the predictive knowledge into the metaheuristic variation operators (ii ) generating new solutions around the estimated optimum; and (iii ) biasing the search by increasing the fitness evaluation inversely proportional to the distance of the current solution to the predictive optimum, discounted by the amount of uncertainty in the prediction.
The latest work found in the literature applying KF estimation in metaheuristics had just been published by the time this thesis was being wrapped up and is due to Muruganantham et al. [159], who proposed tracking each candidate solution in the decision space of a Multi- Objective Optimization (MOO) solver in time-varying – yet deterministic – environments. The general idea somewhat resembles our proposal for tracking mutually non-dominated decision vec- tors presented in chapter 6, although, in our case, the estimation is done in the (d − 1)-simplex and, thus, the linear Gaussian-Markov assumption does not hold, what leads to the need for a different Bayesian estimation model based on the Dirichlet distribution.