Exercises 5.1
1. Suppose S= {0,1}Lis the source alphabet. What are the possible leaves in the parsing of files by S?
2. Show that each of the following does not possess the SPP.
(a) S= {0,01,011,0111,1111}. [Hint: let W = 01.]
(b) S= {00,11,01}
(c) S= {0,1}L\ {w}, for any binary word w of length L.
3. Invent a situation, with a source alphabet S= {s1,...,sm} satisfying the SPP, an encoding scheme si→ wj, j= 1,...,m, satisfying the prefix con-dition (see the next section), and an original file W , such that the compres-sion ratio achieved by parsing and then encoding W is at least 5 to 1. [Hint:
you could take the example of this section as a model, with s1= 0, s2= 10, ... , with an encoding scheme in the mode of the example, and with a silly W ; what if W has all 1’s?]
4. Find the compression ratio if the original file in the example in this section is parsed using S= {0,1}3and encoded using the scheme
000→ 1111111 100→ 1110 001→ 1111110 101→ 110 010→ 111110 110→ 10 011→ 11110 111→ 0
5.2 Review of the prefix condition
We collect here some of the definitions and results fromChapter 4and apply them to our current purpose, the characterization of lists of binary words with the strong parsing property.
A binary word u is a prefix of a binary wordw if and only if w = uv for some (possibly empty) wordv. A list w1,...,wm of binary words satisfies the prefix condition if and only if whenever 1≤ i, j ≤ m and i = j, then wi is not a prefix ofwj. An encoding scheme si → wi, i= 1,...,m, satisfies the prefix condition if and only if the listw1,...,wmdoes. It is common usage to say that such a scheme defines a prefix-condition code, or simply a prefix code. Take note that this terminology is somewhat misleading, because “prefix code” is characterized by an absence of prefix relations in its encoding scheme.
The practical importance of prefix codes is encapsulated in Theorem 4.2.1, which says:
5.2.1 Every prefix code is uniquely decodable, with reading-left-to-right serv-ing as a valid decoder-recognizer.
In case you have not read Section 4.1, here is a translation:
5.2.2 Ifsj → wj, j= 1,...,m, is a prefix-condition encoding scheme, and if for some subscripts1≤ i1,...,it, j1,..., jr ≤ m,wi1···wit = wj1···wjr, then t = r andik = jk, k= 1,...,t. Furthermore, the source word si1···sit can infallibly be recovered from the code wordwi1···wit by scanning left to right and noting the source lettersik each time a wordwikfrom the encoding scheme is recognized.
(Actually, the assertion that “reading-left-to-right is a valid decoder-recog-nizer” for a code given by a particular scheme is a stronger statement than is given in the last part of 5.2.2, because of the word “recognizer”—the last as-sertion in 5.2.2 says that reading-left-to-right is a valid decoder for a prefix code—but we will not tarry further over this point.)
In the example in the preceding section, the encoding scheme(∗∗) satisfies the prefix condition, and if you followed the example you experienced directly the pleasures of reading-left-to-right with respect to the scheme(∗∗). This sort of decoding is also called instantaneous decoding, for reasons that should be obvious.
Pretty clearly, 5.2.2 is telling us that lists of words with the prefix condition satisfy something resembling the uniqueness provision of the strong parsing property. We will give the full relation between the prefix condition and the SPP after restating the Kraft and McMillan Theorems for binary codes. More general statements and proofs are given in Section 4.2.
5.2.3 Kraft’s Theorem for binary codes Supposemand1,...,m are posi-tive integers. There is a listw1,...,wm of binary words, satisfying the prefix condition andlgth(wi) = i,i= 1,...,m, if and only ifmi=12−i ≤ 1. 5.2.4 McMillan’s Theorem for binary codes Suppose m and 1,...,m are positive integers, and w1,...,wm are binary words satisfying lgth(wi) = i, i= 1,...,m. Ifsi→ wi,i= 1,...,mis a uniquely decodable encoding scheme (see the first part of 5.2.2 for the meaning of this) thenmi=12−i ≤ 1.
McMillan’s Theorem has virtually no practical significance at this point.
We repeat it here just because it is a beautiful theorem. Together with Kraft’s Theorem, what it says of pseudo-practical importance is that if your primary criteria for goodness of an encoding scheme are unique decodability, first, and prescribed code word lengths, second, then there is no need to leave the friendly family of prefix codes.
Kraft’s Theorem brings us to the main result of this section. The proof is outlined in Exercise 5.2.2.
5.2.5 Theorem A listw1,...,wmof binary words has the strong parsing prop-erty if and only if the list satisfies the prefix condition andmi=12−lgth(wi)= 1.
5.2 Review of the prefix condition 125
Exercises 5.2
1. Complete the following to lists with the SPP, adding as few new words to the lists as possible.
(a) 00, 01, 10.
(b) 00, 10, 110, 011.
(c) 0, 10, 110, 1110.
2. Prove Theorem 5.2.5 by completing the following.
(a) Show that if binary wordsw1,...,wmdo not satisfy the prefix condition, then the list cannot possess the SPP. [Ifwi = wjv, i = j, then wi = wjv can be parsed in at least two different ways byw1,...,wm.]
(b) Prove this part of the assertion in 5.2.2: Ifw1,...,wm satisfy the pre-fix condition, and ifwi1···wit = wj1···wjr, then r= t and ik = jk, k = 1,...,t. [Hint: if not, let k be the smallest index such that ik= jk. Then wik···wit = wjk···wjr; since the two strings agree at every position, they must agree in the first min(lgth(wik), lgth(wjk)) places. But then w1,...,wm
do not satisfy the prefix condition, contrary to supposition. Why don’t they?]
(c) Supposew1,...,wm are binary words satisfying the prefix condition, andm
i=12−i < 1, where i= lgth(wi), i = 1,...,m. Let ≥ maxii be sufficiently large that 2−+m
i=12−i ≤ 1. By the proof of Kraft’s Theo-rem (seeSection 4.2), there is a binary word W of length that has none of w1,...,wm as a prefix. Conclude that W cannot be parsed into a string wi1···witv for some indices i1,...,it and leave v satisfying lgth(v) <
maxilgth(wi) (why not?) and that, therefore, w1,...,wm does not have the SPP.
(d) Supposew1,...,wm are binary words satisfying the prefix condition andm
i=12−i = 1, where i = lgth(wi), i = 1,...,m. If W is a binary word of length ≥ maxii, then W must have one of thewi as a prefix, for, if not, then the list w1,...,wm,W satisfies the prefix condition, yet 2−lgth(W )+m
i=12−lgth(wi)= 2−+1 > 1, which is impossible, by Kraft’s Theorem. Conclude that every such W can be written W= wi1···witv for some indices i1,...,it andv satisfying lgth(v) < maxii. [How do you arrive at this conclusion?]
Put (a)–(d) together for a proof of Theorem 5.2.5.
3. What are the analogues of the SPP and Theorem 5.2.5 for non-binary code alphabets?
4. Show that any listw1,...,wk of binary words satisfying the prefix condi-tion can be completed to a listw1,...,wmwith the SPP. [You will probably need to use a result, mentioned in part (c) of Exercise 2, above, which is embedded in the proof of Kraft’s Theorem. Incidentally, the result of this exercise holds for non-binary alphabets, as well.]