The term “classical information theory” is used in the literature, by and large, to refer to the theory based on the notion of probability (Chapter 3). Uncer- tainty functions in this theory are expressed in terms of classical measure theory, which in turn, is formalized in terms of classical set theory. Generaliz- ing the concept of a classical measure is thus one way of enlarging the frame- work for a broader treatment of the concept of uncertainty and the associated concept of uncertainty-based information. The purpose of this chapter is to discuss this generalization. Further enlargement of the framework, which is discussed in Chapter 8, is obtained by fuzzifications of classical as well as gen- eralized measures. Basic characteristics of both these generalizations are depicted in Figure 4.1.
Given a universal set Xand a nonempty family Cof subsets of Xwith an appropriate algebraic structure (e.g., a s-algebra), a classical measure,m, is a set function of the form
Uncertainty and Information: Foundations of Generalized Information Theory, by George J. Klir © 2006 by John Wiley & Sons, Inc.
m:CÆ[0,•] that satisfies the following requirements:
(cm1) If ∆ ŒC(familyCis usually assumed to contain ∆), then m(∆)=0; (cm2) For every sequence A1,A2, . . . of pairwise disjoint sets of C,
Observe that probability is a classical measure such that Cis a s-algebra and
m(X)=1.
Property (cm2), which is the distinguishing feature of classical measures, is called a countable additivity. A variant of this property, which is called a finite additivity, is defined as follows:
(cm2¢) for every finite sequence A1,A2, . . . ,An, of pairwise disjoint sets of C, if Ai then A A i n i i n i i n = Œ = = Ê ËÁ ˆ¯˜ =
Â
( ) 1 1 1U
C mU
m . if Ai then A A i i i i i = • = • = • Œ ÊËÁ ˆ¯˜ =Â
( ) 1 1 1U
C mU
m . Boolean algebras: Classical sets or propositions Weaker algebras: Fuzzy sets or propositionsof special types Classical information theory Generalized information theory Classical Measures: Additive set functions
Generalized Measures: Monotone set functions
with special properties
Generalizations
∑ ∑ ∑
∑ ∑ ∑
∑ ∑ ∑
It is well known that any countable additive measure is also finitely additive, but not the other way around.
The requirement of additivity (countable or finite) of classical measures is based on the assumption that disjoint sets are noninteractive with respect to the measured property. This assumption is too restrictive in some application contexts. Consider, for example, a set of workers in a workshop whose purpose is to manufacture products of a specific type. Assume that the set is partitioned into subsets (working groups) A1,A2, . . . ,An, and let m(Ai) denote the number of products made by group Ai(iŒ⺞n) within a given unit of time. Then, clearly, any of the following can happen for any two groups Ai,Aj:
• m(Ai»Aj)=m(Ai)+m(Aj) when groups AiandAjwork separately. • m(Ai»Aj)>m(Ai)+m(Aj) when the groups work together and their co-
operation is efficient.
• m(Ai»Aj)<m(Ai)+m(Aj) when the groups work together and their co-
operation is inefficient.
Numerous other examples could be presented to illustrate that the addi- tivity requirement of classical measures severely limits their applicability. Some examples, relevant to the various issues of uncertainty formalization, are discussed later in this chapter.
After recognizing that classical measures are too restrictive, it is not obvious how to generalize them. One possibility is to eliminate the additivity re- quirement and define generalized measures solely by the requirement (cm1). Although this sweeping generalization seems too radical, it has been found useful in some applications. However, its utility for dealing with uncertainty is questionable. Another possibility is to replace the additivity requirement with an appropriate weaker requirement. It is generally recognized that the highest generalization of classical measures that is meaningful for formalizing uncer- tainty functions is the one that replaces the additivity requirement with a weaker requirement of monotonicity with respect to the subsethood ordering. Generalized measures of this kind are called monotone measures. The follow- ing is their formal definition.
Given universal set Xand nonempty family Cof subsets X(usually with an appropriate algebraic structure), a monotone measure,m, on ·X,CÒis a func- tion of the type
that satisfies the following requirements: (m1) m(∆)=0 (vanishing at the empty set).
(m2) For all A, B ŒC, if AB, then m(A)£m(B) (monotonicity). (m3) For any increasing sequence A1A2. . . of sets in C,
(m4) For any decreasing sequence A1A2. . . of sets in C,
Observe that the same symbol,m, is used for both monotone and additive measures. This does not create any notational confusion since additive mea- sures are contained in the class of monotone measures. It is just required that the meaning of the symbol be stated explicitly when it stands for some special type of monotone measures, such as additive measures.
Functions that satisfy requirements (m1), (m2), and either (m3) or (m4) are equally important in the theory of monotone measures. In fact, they are essen- tial for formalizing imprecise probabilities (Section 4.3). These functions are calledsemicontinuousfrom below or above, respectively. When the universal setXis finite, requirements (m3) and (m4) are trivially satisfied and may thus be disregarded. If XŒCandm(X)=1,mis called a regularmonotone measure (orregularsemicontinuous monotone measure). Uncertainty functions of any type are always regular monotone measures.
Observe that requirement (m2) defines measures that are actually monot- one increasing.By changing the inequality m(A)£m(B) in (m2) to m(A)≥m(B), we can define measures that are monotone decreasing. Both types of monot- one measures are useful, even though monotone increasing measures are more common in dealing with uncertainty. Unless specified otherwise, the term “monotone measure” is used in this book to refer to monotone increasing measures that are regular. The utility of monotone decreasing measures is dis- cussed later in the book.
The following inequalities hold for every monotone measure m: if A,B, A»BŒC, then
(4.1) (4.2) These inequalities follow from monotonicity of m and from the facts that A«BAandA«BB, and similarly,A»BAandA»BB. If, in addition, either the inequality
(4.3) or the inequality (4.4) m(A»B)£m( )A +m( )B m(A»B)≥m( )A +m( )B m(A»B)≥max{m( ) ( )A,m B}. m(A«B)£min{m( ) ( )A,mB},
if Ai then lim A A continuity from above
i i i i = • Æ• = • Œ ( )= ÊËÁ ˆ¯˜ 1 1 1
I
C, m mI
( ).if Ai then lim A A continuity from below
i i i i i = • Æ• = • Œ ( )= ÊËÁ ˆ¯˜ 1 1
U
C, m mU
( ).holds for all A,B,A»BŒCsuch that A«B= ∆, the monotone measure is calledsuperadditiveorsubadditive, respectively.
It is easy to see that additivity implies monotonicity, but not the other way around. For all A,B,A»BŒCsuch that A«B= ∆, a monotone measure m is capable of capturing any of the following situations:
(a) m(A » B) > m(A) + m(B), which expresses a cooperative action or synergy between AandBin terms of the measured property.
(b) m(A»B)= m(A) + m(B), which expresses the fact that Aand Bare noninteractive with respect to the measured property.
(c) m(A»B)<m(A)+m(B), which expresses some sort of inhibitory effect or incompatibility between AandBas far as the measured property is concerned.
Observe that probability theory, which is based on classical measure theory, is capable of capturing only situation (b). This demonstrates that the theory of monotone measures provides us with a considerably broader framework than probability theory for formalizing uncertainty. As a consequence, it allows us to capture types of uncertainty that are beyond the scope of probability theory.
The need for monotone measures arises in many problem areas. One example is the area of ordinary measurement in physics. While additivity char- acterizes well many types of measurement under idealized, error-free condi- tions, it is not fully adequate to characterize most measurements under real, physical conditions, when measurement errors are unavoidable. To illustrate this claim by an example, consider two disjoint events,A and B, defined in terms of adjoining intervals of real numbers, as shown in Figure 4.2a. Obser- vations in close neighborhoods (within a measurement error) of the end points
Discount rate functions Discount rate functions 1
0
Event A Event B Event A B
(a) (b)
(
(
of each event are unreliable and should be properly discounted, for example, according to the discount rate functions shown in Figure 4.2a. That is, obser- vations in the neighborhoods of the end points should carry less evidence than those outside them. The closer they are to the end points, the less evidence they should carry. When measurements are taken for the union of the two events, as shown in Figure 4.2b, one of the discount rate functions is not applic- able. Hence, the same observations produce more evidence for the single event A»Bthan for the two disjoint events AandB. This implies that the proba- bility of A » B should be greater than the sum of the probabilities of A and B. The additivity requirement is thus violated. To properly formalize this situation, we need to use an appropriate monotone measure that is superadditive.
For some historical reasons of little significance, monotone measures are often referred to in literature as fuzzy measures. This name is somewhat con- fusing, since no fuzzy sets are involved in the definition of monotone measures. To avoid this confusion, the term “fuzzy measures” should be reserved to mea- sures (additive or nonadditive) that are defined on families of fuzzy sets.
Since all monotone measures discussed in the rest of this book are regular, it is reasonable to omit the adjective “regular.” Therefore, by convention, the term “monotone measure” refers in the rest of this book to regular monotone measures. Moreover, it is assumed, unless it is stated otherwise, that the uni- versal set,X, is finite and that C=P(X). That is, it is normally assumed that the monotone measures of concern are set functions
whereXis a finite set, that satisfy the following requirements: (m1¢) m(∆)=0 and m(X)=1.
(m2¢) For all A, B ŒP(X), if AB, then m(A)£m(B).