DIFICULTADES DE SURGIDAS/ REFLEXIÓN Y REFORMULACIÓN

In Section 3.7 we explained how a neuro-fuzzy framework can be formulated for the selection of cases (prototypes) under the supervised mode. The present section deals with the methodology of Pal et al. [64], demonstrating an integration of rough sets, fuzzy sets, and self-organizing networks for extracting class prototypes (repre- sentative cases for the entire data set) under the unsupervised mode of learning. One may note here that a similar integration of these three soft computing tools under supervised mode was mentioned in Section 2.3.3, where it is shown how different class regions can be represented with fuzzy membership functions of a varying number for generating representative cases in terms of only informative regions and relevant features (i.e., reduced subsets of original attributes)—thereby enabling fast case retrieval.

Rough set theory [74] provides an effective means for classificatory analysis of data tables. The main goal of rough set–theoretic analysis is to synthesize or con- struct approximations (upper and lower) of concepts from the acquired data. The key concepts here are those of information granule and reducts.Information granule

formalizes the concept of finite-precision representation of objects in real-life situa- tions, and thereductsrepresent the core of an information system (in terms of both objects and features) in a granular universe. An important use of rough set theory has been in generating logical rules for classification and association [75]. These logical rules correspond to different important granulated regions of the feature space, which represent data clusters. For application of rough sets in pattern recog- nition and data mining problems, one may refer to the recent special issue [76].

A self-organizing map (SOM) [77] is an unsupervised network that has recently become popular for unsupervised mining of large data sets. The process of self- organization generates a network whose weights represent prototypes of the input data. These prototypes may be considered as cases representing the entire data set. Unlike those produced by existing case generation methodologies, they are not just a subset of the original data but evolved in the self-organizing process. Since SOM suffers from the problem of slow convergence and local minima, a synergistic inte- gration of rough set theory with SOM offers a fast and robust solution to the initi- alization and local minima problem, thereby designing rough SOM (RSOM). Here rough set theory is used to encode the domain knowledge in the form of crude rules, which are mapped for initialization of weights as well as for determination of the network size. Fuzzy set theory is used for discretization of feature space. Perfor- mance of the network is measured in terms of learning time, representation error, clustering quality, and network compactness. All these characteristics have been demonstrated experimentally and compared with that of the conventional SOM.

3.8.1 Pattern Indiscernibility and Fuzzy Discretization of Feature Space

A primary notion of rough set is of indiscernibility relation. For continuous-valued attributes, the feature space needs to be discretized for defining indiscernibility

relations and equivalence classes. Discretization is a widely studied problem in rough set theory, and fuzzy set theory was used for effective discretization. Use of fuzzy sets has several advantages over ‘‘hard’’ discretization, such as modeling of overlapped clusters and linguistic representation of data [78]. Here each feature is discretized into three levels: low, medium, and high; finer discretizations may lead to better accuracy at the cost of higher computational load.

As mentioned in Sections 2.3.2 and 3.4.2, each feature of a pattern is described in terms of their fuzzy membership values in the linguistic property sets ‘‘low’’ (L), ‘‘medium’’ (M), and ‘‘high’’ (H). Let these be represented byLj,Mj, andHj, respec- tively. The features for the ith pattern Fi are mapped to the corresponding three- dimensional feature space ofmlowðFi;jÞðee^iÞ,mmediumðFi;jÞð^eeiÞ, andmhighðFi;jÞð^eeiÞby equa-

tion (3.33). Ann-dimensional pattern^eei¼ ½Fi1;Fi2;. . .;Finis thus represented as a 3n-dimensional vector [42].

This effectively discretizes each feature into three levels. Then consider only those attributes that have a numerical value greater than some threshold Th (¼0.5, say). This implies clamping only those features demonstrating high mem- bership values with unity while the others are fixed at zero. An attribute-value table is constructed comprising the binary-valued 3n-dimensional feature vectors above. Here we mention that the representation of linguistic fuzzy sets by-functions and the procedure in selecting the centers and radii of the overlapping fuzzy sets are the same as in Section 2.3.3.1. The nature of the functions is also the same as in Figure 2.13.

3.8.2 Methodology for Generation of Reducts

After the binary membership values are obtained for all the patterns, the decision table has been constituted for rough set rule generation. Let there be m sets

O1;O2;. . .;Om of objects in the attribute-value table (obtained by the procedure

described in Section 3.8.1) having identical attribute values, and cardðOiÞ ¼

nki;i¼1;2;. . .;m such that nk1nk2 nkm and

Pm

i¼1nki ¼nk. As in

Section 2.3.3.3, the attribute–value table is represented as an m3n array. Let

nk₁0;nk₂0;. . .;nk_m0 denote the distinct elements among nk1;nk2;. . .;nkm such that

nk₁0 >nk₂0 > >nk0_m Let a heuristic threshold Tr be defined as in equation (2.14), so that all entries having frequency less than Tr are eliminated from the table, resulting in the reduced attribute–value table ^SS. Note that the main motive of introducing this threshold function lies in reducing the size of the model. One attempts to eliminate noisy pattern representatives (having lower values ofnki) from the reduced attribute–value table. From the reduced attribute– value table obtained, reducts are determined using the methodology described below.

Letfxi1;xi2;. . .;xipgbe the set of those objects ofUthat occur in^SS. Now a dis-

cernibility matrix [denotedM(B)] is defined as follows [64,79]:

cij¼ fAttr2B : AttrðxiÞ 6¼AttrðxjÞg for i;j¼1;2;. . .;n ð3:71Þ

For each objectxj2 fxi1;xi2;. . .;xipg, the discernibility functionfxj is defined as fxj ¼ ^f_ðcijÞ : 1i;jn;j<i;cij6¼fg ð3:72Þ

where_ðcijÞis the disjunction of all members ofcij. One thus obtains a ruleri, that is,gi!clusteri, wheregiis the disjunctive normal form offxj;j2i1;i2;. . .;ip. 3.8.3 Rough SOM

3.8.3.1 Self-Organizing Maps A Kohonen feature map is a two-layered net-

work. The first layer of the network is the input layer. The second layer, called the

competitive layer, is usually organized as a two-dimensional grid. All interconnec- tions go from the first layer to the second as in Figure 3.7. For details, see Appendix B. All the nodes in the competitive layer compare the inputs with their weights and compete with each other to become the winning unit having the lowest difference. The basic idea underlying competitive learning is roughly as follows: Assume a sequence of input vectors {^ee¼^eeðtÞ 2Rn, where t is the time coordinate} and a set of variable reference vectors {wiðtÞ : wi2Rn;i¼1;2;. . .;k, where k is the number of units in the competitive layer}. Initially, the values of the reference vec- tors (also calledweight vectors) are set randomly. At each successive instant of time

t, an input pattern ^eeðtÞis presented to the network. The input pattern^eeðtÞis then compared with eachwiðtÞand the best-matchingwiðtÞis updated to match the cur- rent^eeðtÞeven more closely. If the comparison is based on some distance measure

dð^ee;wiÞ, alteringwimust be such that ifi¼c, the index of the best-matching refer- ence vector, then dð^ee;wcÞ is reduced and all the other reference vectors wi, with

i6¼c, are left intact. In this way the various reference vectors tend to become spe- cifically ‘‘tuned’’ to different domains of the input variable^ee.

The first step in the operation of a Kohonen network is to compute a matching value for each unit in the competitive layer. This value measures the extent to which

N×N

grid Competitive layer

Unit 2 Unitn

Unit 1 Input layer

the weights or reference vectors of each unit match the corresponding values of the input pattern. The matching value for each unitiisk^eewik, which is the distance between vectors^eeandwi and is computed by

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi X j ð^eejwijÞ 2 s for j¼1;2;. . .;n ð3:73Þ

The unit with the lowest matching value (the best match) wins the competition. In other words, the unitcis said to be the best-matched unit if

k^eewck¼minifk^eewikg; i¼1;2;. . .;ðNNÞ ð3:74Þ

where the minimum is taken over all theNN units in the competitive layer. If two units have the same matching value, then by convention, the unit with the lower index value iis chosen.

The next step is to self-organize a two-dimensional map that reflects the distri- bution of input patterns. In biophysically inspired neural network models, corre- lated learning by spatially neighboring cells can be implemented using various kinds of lateral feedback connections and other lateral interactions. Here the lateral interaction is enforced directly in a general form, for arbitrary underlying network structures, by defining a neighborhood set Nc around the winning cell. At each learning step, all the cells within Nc are updated, whereas cells outside Nc are

left intact. The update equation is

wij¼

lð^eejwijÞ if uniti is in the neighborhoodNc

0 otherwise

ð3:75Þ

and

wnew_ij ¼wold_ij þwij ð3:76Þ Herelis the learning parameter. This adjustment results in both the winning unit and its neighbors, having their weights modified, becoming more like the input pat- tern. The winner then becomes more likely to win the competition should the same or a similar input pattern be presented subsequently.

3.8.3.2 Incorporation of Rough Sets in SOM As described in Section

3.8.2, the dependency rules generated using rough set theory from an information system are used to discern objects with respect to their attributes. However, the dependency rules generated by rough set are coarse and therefore need to be fine-tuned. Here the dependency rules are used to get a crude knowledge of the cluster boundaries of the input patterns to be fed to a self-organizing map. This crude knowledge is used to encode the initial weights of the nodes of the map, which is then trained using the usual learning process (Section 3.8.3.1). Since an

initial knowledge of the cluster boundaries is encoded into the network, the learning time is reduced greatly, with improved performance.

The steps involved in the process, as formulated by Pal et al. [64], are summar- ized below.

Step 1. From the initial data set, use the fuzzy discretization process to create the information system.

Step 2. For each object in the information table, generate the discernibility function

fAða1;a2;. . .;a3nÞ ¼ ^f_cij : 1i;jn;j<i;cij6¼ ;g ð3:77Þ

wherea1;a2;. . .;a3n are the 3n Boolean variables corresponding to the attributes

Attr1;Attr2;. . .;Attr3n of each object in the information system. The expression

fA is reduced to the set of all prime implicants offA that determines the set of all reducts ofA.

Step 3. The self-organizing map is created with 3n inputs (Section 3.8.1), which correspond to the attributes of the information table, and a competitive layer of

NN grid of units where N is the total number of implicants present in discernibility functions of all the objects of the information table.

Step 4. Each implicant of the function fA is mapped to a unit in the competitive layer of the network, and high weights are given to those links that come from the attributes, which occur in the implicant expression. The idea behind this is that when an input pattern belonging to an object, sayOi, is applied to the inputs of the network, one of the implicants of the discernibility function ofOiwill be satisfied, and the corresponding unit in the competitive layer will fire and emerge as the winning unit. All the implicants of an objectOiare placed in the same layer, while the implicants of different objects are placed in different layers separated by the maximum neighborhood distance. In this way the initial knowledge obtained with rough set methodology is used to train the SOM.

This is explained with the following example. Let the reduct of an objectOibe

Oi : ðF1low^F2mediumÞ _ ðF1high^F2highÞ

whereF_ðÞlow;FðÞmedium;and FðÞhighrepresent the low, medium, and high values of

the corresponding features. Then the implicants are mapped to the nodes of the layer as shown in Figure 3.8. Here high weightsðHÞare given only to those links that come from the features present in the implicant expression. Other links are given low weights.

3.8.4 Experimental Results

Let us explain here some of the results of investigation [64] demonstrating the effectiveness of the methodology on an artificially generated data set (Fig. 3.9)

and two sets of real-life data, the speech data ‘‘vowel’’ and the medical data (which are described in Section 3.7.2.4). The artificial data of Figure 3.9 consist of two features containing 417 points from two horseshoe-shaped clusters. Vowel data has three features and 871 samples from six classes, while the medical data have nine features and deals with hepatobiliary disorders of 536 patients from four classes.

The following quantities were considered for comparing the performance of the RSOM with that of the randomly initialized self-organized map.

H H H H

F1low F1medium F1high F2low F2medium F2high

Figure 3.8 Mapping of reducts in the competitive layer of RSOM.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Feature 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 F eature 2

Figure 3.9 Horseshoe data.

1. Quantization error. The quantization error ðqEÞ measures how fast the weight vectors of the winning nodes in the competitive layer are aligning themselves with the input vectors presented during training. It is calculated using the equation

qE¼

Pn p¼1

P

all wining nodes

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi P jðxpjwjÞ2 q h i number of patterns ð3:78Þ

Herej¼1;2;. . .;m(mbeing the number of input features to the net),xpjis thejth

component of the pth pattern, and n is the total number of patterns. Hence, the higher the quantization errorðqEÞ, the greater the difference between the reference vectors and the input vectors of the nodes in the competitive layer.

2. Entropy andb-index.The entropy measure [23] andb-index [80] reflect the quality of the cluster structure.

Entropy:Let the distance between two weight vectorsp andq be

dðp;qÞ ¼ X j xpjxqj maxjminj 2 " #1=2 ð3:79Þ

wherexpjandxqj denote the weight values forp andq, respectively, along thejth direction, andj¼1;2;. . .;m;mbeing the number of features input to the net; maxj and minjare, respectively, the maximum and minimum values computed over all the samples along thejth axis. Let the similarity betweenp andq be defined as

SMðp;qÞ ¼ebdpq ð₃:₈₀Þ

whereb¼ ln 0:5=d, a positive constant such that

SMðp;qÞ ¼ 1 if dðp;qÞ ¼0 0 if dðp;qÞ ¼ 1 0:5 if dðp;qÞ ¼d 8 < :

wheredis the average distance between points computed over the entire data set.

Entropyis defined as E¼ X l p¼1 Xl q¼1 ðSMðp;qÞlogSMðp;qÞ þ ð1SMðp;qÞÞlogð1SMðp;qÞÞÞ ð3:81Þ If the data are uniformly distributed in the feature space, entropy is maximum. When the data have well-formed clusters, the uncertainty is low and so is the entropy.

b-index: Theb-index[80] is defined as b¼ Pk i¼1 Pni p¼1ð^eeip^eeÞ T_ð^ e ei p^eeÞ Pk i¼1 Pni p¼1ð^eeip^eeiÞ T_ð^ e ei p^eeiÞ ð3:82Þ

whereniis the number of points in theithði¼1;2;. . .;kÞcluster;^eeipthepth pattern

ðp¼1;2;. . .;niÞin clusteri;^eeithe mean ofnipatterns of theith cluster,

P

i ni¼n, wherenis the total number of patterns; and^eeis the mean value of the entire set of patterns. Note thatbis nothing but the ratio of the total variation and within-cluster variation. This type of measure is widely used for feature selection and cluster ana- lysis. For a given data andk(number of clusters) value, the greater the homogeneity within the clustered regions, the higher thebvalue would be.

3. Frequency of winning nodesðfkÞ. This frequency is defined as the number of