FINAL Partido

In many situations, exact lifting is either not applicable or not well suited. In other settings we are required to repeatably lift a network and want to make use of the previous runs of CP. For these and other scenarios, advanced lifting techniques have been developed and are of interest of current research. We will here discuss some these approaches.

Approximate Lifting

In many applications, in particular if LBP is not applied to relational models without evidence, the underlying factor graph is not perfectly symmetric. For this reason, different approaches to approximate lifting have been proposed. The most simple way of approximating the exact lifting is by early stopping the CP.

Instead of running CP until convergence, we stop the exchange of new colors early. By doing so, we prevent that the model is completely shattered to the ground case. It is interesting to note that early stopping can lead to fractional counts. For example, after running CP on

X1 f1 X2 f2 X3 f3 X4 f4 X5

Figure 2.8: A chain-MRF leads to fractional counts when Color Passing is stopped early

the graph shown in Figure 2.8 for two iterations, we obtain for the variables a coloring of (1, 2, 2, 2, 1), similarly we obtain the coloring of (1, 2, 2, 1) for the factors. If we now extract the lifted graph from this coloring, we see that we obtain fractional counts. This is due to the fact that not all variables in the cluster with color 2 are connected to factors with the same color. This information has not been propagated yet. To be more precise, variables X2, X3, and, X4 have the same colors but are connected in some cases to a factor with color 1 and in other cases to factors of color 2. In the end, this leads to a count of 2/3 for f1 and 4/3 for f2. However, the sum of the counts is sill correct, i.e., 2.

Akin to early stopping based on CP, de Salvo Braz et al. [47] presented Anytime LBP. Anytime LBP follows the top-down lifting paradigm and shatters the model iteratively. Starting from a query predicate, Anytime LBP adds neighboring predicates of the query in each iteration to the model which can be seen as the unrolling of the computation tree. This can be done on the lifted level until newly added neighbors require a shattering.

In [115], we presented an approach to approximate lifting which also refines the lifting in each iteration of inference and is close to the approach by de Salvo Braz et al. [47]. However, our approach works in a bottom-up fashion and is again independent of a first-order formalism. Intuitively, the proposed algorithm informed Lifted Belief Propagation (iLBP) interleaves lifting with BP iterations. More precisely, iLBP first runs a single iteration of CP on the ground graph and this is followed by a single iteration of BP on the resulting lifted graph. The graph is now lifted again based on the BP messages of this first iteration, i.e., variables and factors are clustered together if their difference in beliefs is small. This process is iteratively repeated until now new clusters arise. Essentially, iLBP is replacing CP’s syntactic criterion for comparison by a real-valued similarity measure. iLBP is particularly well suited in situations where standard LBP is not efficient because the CP poses an overhead. This can be for example the case when BP converges in very few iterations. In iLBP, nodes can be grouped together that have different computation trees but similar beliefs. By adjusting the definition of “difference in messages”, we can force the algorithm to produce more or fewer clusters. iLBP finds smaller factor graphs, hence further speeding up inference, without a decrease in performance, as it has been shown empirically on different datasets. For example. iLBP has been successfully used on the content distribution problem in peer-to-peer networks.

More recently, Singla et al. [211] further analyzed early stopping in their top-down lifting approach from [209] and also introduced noise-tolerant hypercubes. Similar to iLBP, the noise-tolerant hypercubes allow for marginal errors in the clusternodes by forming coarser factor graphs. I.e., variables are clustered together that are separated in the original lifted BP approach. In the framework of bisimulation, Sen et al. [198] use binning of factors that are within an user-specified distance to approximate lifting.

It has been shown in [4] that shattering a large graph into pieces gives opportunities for lifting in an online training settings. This piecewise approach breaks long-range dependencies and removes asymmetries that prevented lifting of the entire graph. Instead of training on the entire graph, the pieces are used as mini-batches and lifting can be applied to these.

Online Lifted Inference

In many applications one has to apply lifted inference to a sequence of problems with only little differences between each of the subproblems. Here, it is desirable to avoid lifting from scratch for each subproblem. One particular instance of such a setting is the case where only the evidence changes in each subproblem. A few algorithms have been proposed that try to become more efficiently by exploiting symmetries across inference instances [2, 3, 165]. So far, these algorithms have concentrated to efficiently obtain a lifted data structure for subsequently following inference instances with changes in knowledge, i.e., evidence. We will make use of such ideas and extend these for our purposes in Chapter 3. In contrast to existing approaches approaches, we will show in Section 3.3 how changing evidence during a single inference run can be exploited as well.

We have seen in this section how probabilistic inference algorithms can benefit from lifting. In particular, we have seen how BP can be lifted by using modified update equations running on a compressed factor graph. Before we will present our contributions in form of new lifted message passing algorithms and modified lifting procedures in Chapter 3, we will briefly summarize quality measures that will be used throughout this thesis for comparison and evaluation.