Exercise 1.1 Kerrich [1946] performed experiments such as tossing a coin many times, and he found that the relative frequency did appear to approach a limit. That is, for example, he found that after 100 tosses the relative frequency may have been .51, after 1000 it may have been .508, after 10, 000 tosses it may have been .5003, and after 100, 000 tosses, it may have been .50008. The pattern is that the 5 in the first place to the right of the decimal point remains in all relative frequencies after the first 100 tosses, the 0 in the second place remains in all relative frequencies after the first 1000 tosses, etc. Toss a thumbtack at least 1000 times and see if you obtain similar results.
Exercise 1.2 Pick some upcoming event (It could be a sporting event or it could even be the event that you get an ‘A’ in this course.) and determine your probability of the event using Lindley’s [1985] method of comparing the uncertain event to a draw of a ball from an urn (See Example 1.3.).
Exercise 1.3 Prove Theorem 1.1.
Exercise 1.4 Example 1.6 showed that, in the draw of the top card from a deck, the eventQueen is independent of the event Spade. That is, it showed P (Queen|
Spade) = P (Queen).
1. Show directly that the eventSpade is independent of the event Queen. That is, show P (Spade|Queen) = P (Spade). Show also that P (Queen∩Spade) = P (Queen)P (Spade).
2. Show, in general, that if P (E) 6= 0 and P (F) 6= 0, then P (E|F) = P (E) if and only if P (F|E) = P (F) and each of these holds if and only if P (E∩F) = P (E)P (F).
Exercise 1.5 The complement of a setE consists of all the elements in Ω that are not inE and is denoted by E.
1. Show thatE is independent of F if and only if E is independent of F, which is true if and only ifE is independent of F.
2. Example 1.8 showed that, for the objects in Figure 1.2,One and Square are conditionally independent givenBlack and given White. Let Two be the set of all objects containing a ‘2’ and Round be the set of all round objects.
Use the result just obtained to concludeTwo and Square, One and Round, andTwo and Round are each conditionally independent given either Black or White.
Exercise 1.6 Example 1.7 showed that, in the draw of the top card from a deck, the eventE = {kh, ks, qh} and the event F = {kh, kc, qh} are conditionally independent given the eventG = {kh, ks, kc, kd}. Determine whether E and F are conditionally independent givenG.
Exercise 1.7 Prove the rule of total probability, which says if we have n mu-tually exclusive and exhaustive events E1,E2, . . .En, then for any other event F,
P (F) = Xn i=1
P (F ∩ Ei).
Exercise 1.8 Let Ω be the set of all objects in Figure 1.2, and assign each object a probability of 1/13. Let One be the set of all objects containing a 1, andSquare be the set of all square objects. Compute P (One|Square) directly and using Bayes’ Theorem.
Exercise 1.9 Let a joint probability distribution be given. Using the law of total probability, show that the probability distribution of any one of the random variables is obtained by summing over all values of the other variables.
Exercise 1.10 Use the results in Exercise 1.5 (1) to conclude that it was only necessary in Example 1.18 to show that P (r, t) = P (r, t|s1) for all values of r and t.
Exercise 1.11 Suppose we have two random variables X and Y with spaces {x1, x2} and {y1, y2} respectively.
1. Use the results in Exercise 1.5 (1) to conclude that we need only show P (y1|x1) = P (y1) to conclude IP(X, Y ).
2. Develop an example showing that if X and Y both have spaces containing more than two values, then we need check whether P (y|x) = P (y) for all values of x and y to conclude IP(X, Y ).
Exercise 1.12 Consider the probability space and random variables given in Example 1.17.
EXERCISES 61 1. Determine the joint distributions of S and W , of W and H, and the
remaining values in the joint distribution of S, H, and W .
2. Show that the joint distribution of S and H can be obtained by summing the joint distribution of S, H, and W over all values of W .
3. Are H and W independent? Are H and W conditionally independent given S? If this small sample is indicative of the probabilistic relationships among the variables in some population, what causal relationships might account for this dependency and conditional independency?
Exercise 1.13 The chain rule states that given n random variables X1, X2, . . . Xn, defined on the same sample space Ω,
P (x1, x2, . . .xn) = P (xn|xn−1, xn−2, . . .x1) · · · P (x2|x1)P (x1) whenever P (x1, x2, . . .xn) 6= 0. Prove this rule.
Section 1.2
Exercise 1.14 Suppose we are developing a system for diagnosing viral infec-tions, and one of our random variables is F ever. If we specify the possible values yes and no, is the clarity test passed? If not, further distinguish the values so it is passed.
Exercise 1.15 Prove Theorem 1.3.
Exercise 1.16 Let V = {X, Y, Z}, let X, Y , and Z have spaces {x1, x2}, {y1, y2}, and {z1, z2} respectively, and specify the following values:
P (x1) = .2 P (y1|x1) = .3 P (z1|x1) = .1 P (x2) = .8 P (y2|x1) = .7 P (z2|x1) = .9 P (y1|x2) = .4 P (z1|x2) = .5 P (y2|x2) = .6 P (z2|x2) = .5.
Define a joint probability distribution P of X, Y , and Z as the product of these values.
1. Show that the values in this joint distribution sum to 1, and therefore this is a way of specifying a joint probability distribution according to Definition 1.8.
2. Show further that IP(Z, Y |X). Note that this conditional independency follows from Theorem 1.5 in Section 1.3.3.
Exercise 1.17 A forgetful nurse is supposed to give Mr. Nguyen a pill each day. The probability that she will forget to give the pill on a given day is .3. If he receives the pill, the probability he will die is .1. If he does not receive the pill, the probability he will die is .8. Mr. Nguyen died today. Use Bayes’ theorem to compute the probability the nurse forgot to give him the pill.
Exercise 1.18 An oil well may be drilled on Professor Neapolitan’s farm in Texas. Based on what has happened on similar farms, we judge the probability of oil being present to be .5, the probability of only natural gas being present to be .2, and the probability of neither being present to be .3. If oil is present, a geological test will give a positive result with probability .9; if only natural gas is present, it will give a positive result with probability .3; and if neither are present, the test will be positive with probability .1. Suppose the test comes back positive. Use Bayes’ theorem to compute the probability oil is present.
Section 1.3
Exercise 1.19 Consider Figure 1.3.
1. The probability distribution in Example 1.25 satisfies the Markov condition with the DAGs in Figures 1.3 (b) and (c). Therefore, owing to Theorem 1.4, that probability distribution is equal to the product of its conditional distributions for each of them. Show this directly.
2. Show that the probability distribution in Example 1.25 is not equal to the product of its conditional distributions for the DAG in Figure 1.3 (d).
Exercise 1.20 Create an arrangement of objects similar to the one in Figure 1.2, but with a different distribution of values, shapes, and colors, so that, if random variables V , S, and C are defined as in Example 1.25, then the only independency or conditional independency among the variables is IP(V, S). Does this distribution satisfy the Markov condition with any of the DAGs in Figure 1.3? If so, which one(s)?
Exercise 1.21 Complete the proof of Theorem 1.5 by showing the specified con-ditional distributions are the concon-ditional distributions they notationally represent in the joint distribution.
Exercise 1.22 Consider the objects in Figure 1.2 and the random variables defined in Example 1.25. Repeatedly sample objects with replacement to obtain estimates of P (c), P (v|c), and P (s|c). Take the product of these estimates and compare it to the actual joint probability distribution.
EXERCISES 63 Exercise 1.23 Consider the objects in Figure 1.2 and the joint probability dis-tribution of the random variables defined in Example 1.25. Suppose we compute its conditional distributions for the DAG in Figure 1.3 (d), and we take their product. Theorem 1.5 says this product is a joint probability distribution that constitutes a Bayesian network with that DAG. Is this the actual joint probability distribution of the variables? If not, what is it?
Section 1.4
Exercise 1.24 Professor Morris investigated gender bias in hiring in the fol-lowing way. He gave hiring personnel equal numbers of male and female resumes to review, and then he investigated whether their evaluations were correlated with gender. When he submitted a paper summarizing his results to a psychology journal, the reviewers rejected the paper because they said this was an example of fat hand manipulation. Explain why they might have thought this. Elucidate your explanation by identifying all relevant variables in the RCE and drawing a DAG like the one in Figure 1.15.
Exercise 1.25 Consider the following piece of medical knowledge taken from [Lauritzen and Spiegelhalter, 1988]: Tuberculosis and lung cancer can each cause shortness of breath (dyspnea) and a positive chest X-ray. Bronchitis is another cause of dyspnea. A recent visit to Asia can increase the probability of tuber-culosis. Smoking can cause both lung cancer and bronchitis. Create a DAG representing the causal relationships among these variables. Complete the con-struction of a Bayesian network by determining values for the conditional prob-ability distributions in this DAG either based on your own subjective judgement or from data.