to see how it can be generalized to any size image. In principle, the value of a pixel could be any color, but for the sake of example, let’s say it’s a Boolean representing whether the pixel is bright or dark. There’s an edge between any pair of pixels that are adjacent either horizontally or vertically. Intuitively, these edges encode the fact that, all else being equal, two adjacent pixels are more likely to have the same value than different values.
This “all else being equal” qualifier is important to understand the meaning of this model. If you ignore the edge between two pixels for a moment and consider the individual probability distribution over each of the pixels, it might be likely that they are, in fact, different. For example, based on everything else you know, you might believe that pixel 11 is bright with probability 90% and that pixel 12 is bright with probability 10%. In this case, it’s highly likely that pixel 11 and pixel 12 are dif- ferent. But the edge between pixel 11 and pixel 12 makes them more likely to be the same than they would otherwise have been. It adds a piece of knowledge that they’re likely to be the same. This knowledge counterweighs the other knowledge that they’re likely to be different, but it might not completely change the overall conclu- sion. The specific knowledge expressed by the edge between pixel 11 and pixel 12 is represented by the potential on that edge. Now let’s see exactly how potentials are defined.
POTENTIALS
How are the numerical parameters of a Markov network defined? In a Bayesian net- work, each variable has a CPD. In a Markov network, it’s not as simple. Variables don’t own their numerical parameters. Instead, functions called potentials are defined on sets of variables. When there’s a symmetric dependency, some joint states of the vari- ables that are dependent on each other are more likely than others, all else being equal. The potential specifies a weight for each such joint state. Joint states with high weights are more likely than joint states with low weights, all else being equal. The rel- ative probability of the two joint states is equal to the ratio between their weights, again, all else being equal.
Mathematically, a potential is simply a function from the values of variables to real numbers. Only positive real numbers or zero are allowed as the values of a potential. Table 5.1 shows an example of a unary potential over a single pixel for the image- recovery application, and table 5.2 shows a binary potential over two pixels.
Table 5.1 A unary potential over a single pixel. This potential encodes the fact that, all else being equal, a pixel is lit with probability 0.4.
Pixel 31 Potential value
F 0.6
How do potential functions interact with the graph structure? There are two rules: ■ A potential function can mention only variables that are connected in the graph. ■ If two variables are connected in the graph, they must be mentioned together
by some potential function.
In our image-recovery example, every variable will have a copy of the unary potential in table 5.2, and every pair of adjacent pixels, either horizontally or vertically, will have a copy of the binary potential in table 5.2. You can see that the two rules are respected by this assignment of potentials.
HOWA MARKOVNETWORKDEFINESAPROBABILITYDISTRIBUTION
You’ve seen how a Markov network is defined. How does it define a probability distri- bution? How does it assign a probability to every possible world so that the probabili- ties of all possible worlds add up to 1? The answer isn’t quite as simple as for Bayesian networks but also isn’t too complicated.
Just as in a Bayesian network, a possible world in a Markov network consists of an assignment of values to all of the variables, making sure that the value of each variable is in its domain. What’s the probability of such a possible world? Let’s build it up piece by piece by using an example.
To keep things simple, let’s consider a 2 × 2 array of pixels with the following assignment of values: pixel 11 = true, pixel 12 = true, pixel 21 = true, pixel 22 = false. You’ll look at all potentials in the model and their potential values for this possible world. For the unary potentials, you have the values in table 5.3. Pixels that are true Table 5.2 A binary potential over two adjacent pixels. This potential encodes the fact
that, all else being equal, the two pixels are nine times as likely to have the same value as different values.
Pixel 31 Pixel 32 Potential value
F F 0.9
F T 0.1
T F 0.1
T T 0.9
Table 5.3 Potential values for unary potentials for example possible world
Variable Potential value
Pixel 11 0.4
Pixel 12 0.4
Pixel 21 0.4
169