As Figure 4.14 displays, the fuzzy rule-based system is used for both classification and interpretation and only needs the ensemble method support when the certainty grade of its classification is low or in case of rejected and uncovered classifications. But one
University
121
important question can be asked: do we lose the interpretability of the classification when the ensemble method is used for classification instead of the fuzzy rule-based system?.
Actually, we can get the interpretation of a pattern classified using the ensemble method by identifying the winner rule among the set of rules that has the same class label as the class predicted by the ensemble method. We can justify the logic behind this method as the following: since the classification certainty of the fuzzy rule-based system is below the predefined threshold value then we reject the classification. In this case, we need a reliable classifier to help identify the correct classification which is the ensemble method in our case. So, rather than identifying the winner rule among all the classes, as we do when the classification is performed by the fuzzy rule-based system, we limit the competition only among the rules whose class label is the same as the class predicted by the ensemble method. Thus, the winner rule which has the same class label as the ensemble method is used for local interpretation of the classified pattern. The only case in which we lose the local interpretation of a pattern classification is when the class label produced by the ensemble method is not covered by any rule.
We need to mention that the uncovered patterns do not occur only when we apply the ensemble method but even when we use the fuzzy rule-based system for classification.
Rejection methods and threshold calculation
Rejection methods have been used in the literature to define the way in which the classification of a given classifier is rejected (G. Fumera, 2002). The same thing for our case, rejection methods determine when the classification of a fuzzy rule-based system is rejected and thus when the ensemble method is used for classification. We used two commonly rejection methods (Ishibuchi & Nakshima, 1998; Ishibuchi & Nii, 2000) to call the ensemble method. In (Ishibuchi & Nii, 2000), the authors assumed that the threshold is pre-specified. In this study, however, we introduced two methods to
University
122
calculate the threshold value 𝜃 under which the classification is rejected. These methods (used for threshold value 𝜃 calculation) along with their respective rejection methods will be compared in terms of accuracy and local interpretability rates to choose the most suitable one. In what follows is a description of the two methods.
Method1: The threshold 𝜃 1 in this method is calculated as the average product of the compatibility grades and the certainty grades of winner rules that incorrectly classified the training patterns. Threshold 𝜃 1 can be calculated by:
𝜃 1 = ∑ 𝜇𝑅 𝑤 𝑗(𝑥𝑗). 𝑟𝑗𝑤. 𝑙𝑗 𝑚 𝑗=1 ∑𝑚 𝑙𝑗 𝑗=1 (4.14)
Where 𝑟𝑗𝑤 is the certainty grade of the winner rule 𝑅𝑤𝑗 for the training pattern j while 𝜇𝑅
𝑤𝑗(𝑥𝑗) is the compatibility grade of the antecedent part of the winner rule with the pattern 𝑗. 𝑙j = 1 if the fuzzy system misclassified 𝑥𝑗 and 𝑙j= 0 otherwise. 𝑚 is the number of training patterns.
The rejection method that corresponds to Method1 works as follows: when a new pattern 𝑥𝑗 is presented for classification, we calculate the product 𝜇𝑅
𝑤𝑗(𝑥𝑗). 𝑟𝑗
𝑤 and then
we compare it with 𝜃 1. If 𝜇𝑅
𝑤𝑗(𝑥𝑗). 𝑟𝑗
𝑤 > 𝜃
1 then we use the fuzzy rule-based system otherwise we reject the classification and call for the ensemble method to perform the classification instead.
Method2: in this method, the threshold 𝜃 2 is calculated as the following:
𝜃 2 = ∑ (𝜇𝑅𝑤𝑗(𝑥𝑗). 𝑟𝑗 𝑤− 𝜇 𝑅𝑤𝑗′(𝑥𝑗). 𝑟𝑗 𝑤′). 𝑙 𝑗 𝑚 𝑗=1 ∑𝑚𝑗=1𝑙𝑗 (4.15)
Where 𝑟𝑗𝑤′ is the certainty grade of the second best rule 𝑅𝑤𝑗′ whose class label is different from that of the winner rule 𝑅𝑤𝑗 for the training pattern 𝑗 while 𝜇𝑅
𝑤
𝑗′(𝑥𝑗) is the compatibility grade of the antecedent part of the second best rules with the training pattern 𝑗.
University
123
For testing classification, we apply the following rejection method: first we calculate 𝜇𝑅
𝑤𝑗(𝑥𝑗). 𝑟𝑗 𝑤− 𝜇
𝑅𝑤′𝑗 (𝑥𝑗). 𝑟𝑗𝑤′ and then compare it with 𝜃 2. If 𝜇𝑅𝑤𝑗(𝑥𝑗). 𝑟𝑗𝑤 − 𝜇𝑅
𝑤′
𝑗 (𝑥𝑗). 𝑟𝑗𝑤 ′
> 𝜃 2 then we use fuzzy rule-based system otherwise we reject the
classification and call for the ensemble method to classify the pattern 𝑗.
Comparison between Method1 and Method2
A comparison between Method1 and Method2 will be made using two criteria: the accuracy and interpretability. The accuracy is evaluated in terms of testing error rates while the interpretability is assessed using the local interpretability rates. We mean by patterns with local interpretability those which are both correctly classified and covered.
Calculate 𝜇𝑅 𝑤 𝑗(𝑥𝑗). 𝑟𝑗𝑤 of the winner rule 𝜇𝑅 𝑤𝑗(𝑥𝑗). 𝑟𝑗 𝑤> 𝜃 1 No
Calculate the class label using the ensemble classifier
Start
A pattern 𝑗 is presented
Yes Calculate the class label
using FRBS
End
Figure 4.14 Flowchart of fuzzy-ensemble classification with rejection threshold 𝜃 1 associated with Method1
University
124
In the fuzzy rule-based system, all the correctly classified patterns are covered by their winner rules and thus have local interpretation. But in case of the fuzzy-ensemble method, some patterns are classified by the ensemble method and they may not have coverage by any rule even if they are correctly classified by the ensemble method. In this case, we cannot get the interpretation of these patterns. In addition, there is no importance to get the interpretation of misclassified testing patterns. So, in order get a local interpretation of a specific pattern classification output, the classification of that pattern has to be both correct and covered. The local interpretability rate can be calculated using the equation:
𝐿𝑜𝑐𝑎𝑙 𝑖𝑛𝑡𝑒𝑟𝑝𝑟𝑒𝑡𝑎𝑏𝑖𝑙𝑖𝑡𝑦 𝑟𝑎𝑡𝑒 (%) = 100 ×𝑁𝑖𝑛𝑡
𝑚 (4.16) Where 𝑁𝑖𝑛𝑡 is the number of testing patterns that are both correctly classified and covered and 𝑚 is the total number of testing patterns. Equation (4.16) suggests that the local interpretability rate of the fuzzy-ensemble method is less or equal to the testing accuracy rate while they are equal in the case of the fuzzy rule-based system.