In the previous sections, we discussed the GANN performance to handle data with high feature dimension, sample scarcity and complex feature interaction. Based on these findings, we observed that the tanh based system is, amongst all the systems, the most effective system to extract the most significant features from the biology perspective.
In this section, we examine the selection performance of the tanh based system to handle bioassay data characterised by low feature dimension, feature-independent and highly imbalanced between the positive and the negative samples. An experiment was conducted using the bioassay data sets based on the population size 300 and the fitness evaluation size 30000. The completeness of the findings is evaluated using four cost-sensitive classifiers constructed in the WEKA environment (see Section 4.1.2 on page 95 for parameter settings) and is compared with the original work reported by Schierz (2009). The data sets have been split into an 80% training set and a 20% test set, as recommended by Schierz. For AID362 data set, the training set contains 3423 compounds, i.e. 48 active and 3375 inactive; and the test set contains 856 compounds, i.e. 12 active and 844 inactive. For AID688 data set, the training set has 21751 compounds, i.e. 198 active and 21553 inactive; and 5438 compounds, i.e. 50 active and 5388 inactive.
The top 20% of the attributes from each data set were selected by GANN prototype. To evaluate the generalisation performance of the prototype, these selected attributes were trained by the identical set of cost-sensitive classifiers (CSC) as reported in the original study (Schierz, 2009), i.e. Naive Bayes (CSC NB), Support Vector Machine (CSC SMO), C4.5 tree (MetaCost J48) and Random Forest (CSC RF), with a 10- fold cross-validation procedure. The significance of the attributes were then validated using the independent test set.
Standard classifiers usually assume equal weighting of the classes in the data set, in the sense that mis- classifying class A has similar consequences as misclassifying class B. As a result, most standard classifiers are unable to predict a minority class in the bioassay data sets (Maloof, 2003; Schierz, 2009) due to highly imbalanced data between active and inactive compounds in the data set. This led to the development of cost-sensitive classifiers, which can assign different costing for different weighting classes in imbalanced data sets (Domingos, 1999; Elkan, 2001; Maloof, 2003).
A trial experiment, based on the network size 10-5-2 using the AID362 data set was performed and the results showed a poor performance on the system. This was due to the large data size of bioassay data (4279-27189 samples) when compared to microarray data (72-83 samples), thus, we increased the network size of GANN to 20-10-2, while the remaining parameters remain unchanged. Figure 5.9 shows the summary of the results based on the independent test set of the data sets and the list of the selected attributes by
GANN is presented in Table B.16 in Appendix B. We have also conducted the experiments using principal component analysis (PCA).
(a) The true positive (TP) rate of the bioassay data sets with under or approximately a 20% false positive (FP) rate.
(b) The false positive (FP) rate of the bioassay data sets.
Figure 5.9: The classification results based on the independent test set in the bioassay data sets.
There has been no significant decrease in classification performance of the cost-sensitive classifiers, as showed in Figure 5.9a. Using the 28 attributes selected by GANN in the AID362 data set, a CSC RF has produced better results than using the entire 144 attributes reported by Schierz (2009) and the 95 attributes selected by PCA. There has been a slight decrease in performance when using CSC NB and CSC SMO, but all other results are comparable. Furthermore, using the 28 attributes selected by GANN, a significant decrease on the number of false positive (FP) rate was observed in all classifiers, except a CSC SMO, as is presented in Figure 5.9b.
For AID688 data set, as showed in Figure 5.9a, using the 31 attributes selected by GANN, a MetaCost J48 tree has performed better than using the entire 153 attributes reported by Schierz (2009) and the 116 attributes selected by PCA. Surprisingly, the CSC RF which was unable to run using neither the whole
data set nor the 116 attributes selected by PCA, due to the size of the data set (∼ 27000 compounds), has produced good results using the attributes selected by GANN. However, there has been a decrease in performance using CSC NB and CSC SMO for attributes selected by GANN when compared to using the entire attributes and the 116 attributes selected by PCA. Even so, all cost-sensitive classifiers have the lowest FP using the 31 attributes selected by GANN (see Figure 5.9b).
The findings in this section demonstrate the efficiency of the GANN prototype to handle a large, imbalanced data set. The results show a comparable better performance on the GANN prototype and the entire at- tributes in the data set and the PCA. The advantage of using GANN to select attributes over using the entire attributes and the PCA is that GANN enables computationally effective algorithms, such as Random Forest and Classification Tree, to be implemented in a larger data set with a high success rate. Considering the GANN prototype only implemented 20% of the attributes from the data set, the classification performance of cost-sensitive classifiers has not been sacrificed. This shows that the GANN prototype has successfully identified the most significant attributes needed to discriminate between the active and the inactive com- pounds. The only downside of the GANN prototype is that a computationally intensive processing time required for larger data set.
5.8
Summary
In this chapter, we examined the performance of the GANN model as a feature extraction. The prototype was implemented to extract informative features (genes and attributes) from six data sets, comprising two synthetic data, two microarray data and two bioassay data. The results can be summarised as follows:
• The linear based system is able to explore the potential genes more effectively than the other three sys- tems and is very much processing cost effective as compared to the remaining three systems. However, this system also induced a high number of low interest genes in the subject of study.
• The sigmoid based system is also able to efficiently explore the potential genes. However, this system requires a more intensive processing cost than the other three systems.
• The threshold based system is lack of the stability factor in extracting consistent genes and it is sensitive to data distribution.
• The tanh based system is, amongst the four systems, the most effective system to extract the most informative genes from the data sets.
• All systems have a significant improvement on their performance with the population sizes 200 to 300 and the fitness evaluation sizes, ranging from 20000 to 40000.
• The identification of informative genes that were lower expressed in the data set can be achieved with a low fitness precision level. These genes may be useful for therapeutic and biological diagnostic to prevent the development of a tumour.
• The improper use of normalisation technique and a lack of understanding on the microarray data could compromise the integrity of the results.
• The tanh based GANN system has produced better, or at least comparable, results in a large and imbal- anced bioassay data set. However, the only downside of the GANN prototype is that a computationally intensive processing time required for larger data set.
In the next chapter, we conclude our research by revising our contributions to the bioinformatics field, the revision on our methodology and suggests the trend of the future research. The chapter will concluded with the overall achievement of this thesis.
Conclusion and Future Works
This chapter draws conclusions on our work reported in this thesis . A summary of the major contributions of the research is provided and suggestions for possible further research areas.
This chapter contains five sections. Section 6.1 provides conclusions of the thesis. Section 6.2 summarises the major contributions for this research. Section 6.3 presents the areas that have been omitted in this thesis. Section 6.4 indicates the limitations of our research and suggests several interesting avenues for further work to extend our research. Finally, Section 6.5 concludes the thesis with the overall achievements of the research.