Evolutive Optimization of Wavelets and Shapelets for Bioelectrical Signal Analysis

Texto completo

(1)Evolutive Optimization of Wavelets and Shapelets for Bioelectrical Signal Analysis. Ruben-Dario Pinzon-Morales Department of Electrical Engineering Universidad Tecnologica de Pereira. A thesis submitted for the degree of Masters of Science Yet to be decided.

(2) December 15, 2011. Thesis Advisor: Dr. Alvaro-Angel Orozco-Gutierrez, Titular Professor at the Universidad Tecnologica de Pereira. CERTIFY: that the presented master’s thesis entitled “Evolutive Optimization of Wavelets and Shapelets for Bioelectrical Signal Analysis” has been written under his supervision by the Electronic Engineer Ruben-Dario Pinzon-Morales and have not used any sources and aids other than those indicated.. The present certification is issued in Pereira, December of 2011.. Signed. Dr. Alvaro-Angel Orozco-Gutierrez..

(3) Remember –please avoid printing this document where possible to avoid wasting paper, if printed copy is required please print double sided. Take care of the enviorement..

(4) Acknowledgements. And I would like to acknowledge to f : [a, b] ⊆ R → R, a < b ⇒ ∃r, s ∈ [a, b]|f (r) ≤ f (x) ≤ f (s) where f (x) is the lattice of all my friends, classmates, coworkers, bosses and advisors, and the only value for x ∈ [a, b] is my most sincere thanks..

(5) Abstract Wavelet analysis is a powerful tool for digital signal processing. It has been widely used in bioelectric signals including evoke-related potentials (ERP), electromyography signals (EMG), microelectrode recordings (MER), electrocardiogram (ECG), electroencephalograms (EEG), among others. Some of the principal advantages of wavelet transform are the compact support, and energy concentration. Basically, the wavelet transform is a convolution of the input signal with scaled and translated versions of one function called wavelet mother. Although there are plenty of wavelet prototypes in the literature, there is not an established rule that states which wavelet may be used for each application. Instead of that, it is a usual task for the researcher to test more o less arbitrarily different wavelet shapes in order to find one suited. In this manuscript the capabilities of the lifting schemes, which are a technique for both designing wavelets and performing the discrete wavelet transform and genetic algorithms are extended for wavelet synthesis in two scenarios: the signal–dependent filter bank and the discrete lifting shapelet transform. The former is an evolutionary method based on LS that aims at the synthesis of mother wavelets that exhibit unique time–frequency properties. Furthermore, the designed wavelet function is supposed to exhibit feature spaces with maximum class separability due to the employment of the clustering validation metrics as fitness function into the GA procedure. Results presented in three scenarios render the success of the proposed methodology including the identification of basal ganglia by means of microelectrode recording analysis, the recognition of hand movement using electromyographic signals, and the classification of epileptic seizures employing EEG signals..

(6) Publications International Conference Proceedings • Pinzon-Morales, R.-D., Orozco-Gutierrez, A.-A., Castellanos, C.-G. (2010). Feature selection using an ensemble of optimal wavelet packet and learning machine: Application to MER signals. In 2010 7th International Symposium on Communication Systems, Networks and Digital Signal Processing, CSNDSP 2010 (pp. 25–30). • Pinzon-Morales, R.-D., Orozco-Gutierrez, A.-A., Carmona-Villada, H. Castellanos, C.-G. (2010). Towards high accuracy classification of MER signals for target localization in Parkinson’s disease. In Engineering in Medicine and Biology Society (EMBC), 2010 Annual International Conference of the IEEE (pp. 4040 -4043). • Pinzon-Morales, R.-D., Garces-Arboleda M., Orozco-Gutierrez, A.-A. (2009). Automatic identification of various nuclei in the basal ganglia for Parkinson’s disease neurosurgery. In Proceedings of the 31st Annual International Conference of the IEEE Engineering in Medicine and Biology Society: Engineering the Future of Biomedicine, EMBC 2009 (pp. 3473–3476). • Pinzon-Morales, R.-D., Restrepo, F., Moscoso, O., Castro-Cabrera, P.A., Orozco-Gutirrez, A.A., Castellanos-Domnguez, C.G. (2010). Detection of Attention-Deficit/Hyperactivity Disorder based on Customized Wavelet. Proceedings of the XXVIII Congreso Anual de la Sociedad Espaola de Ingeniera Biomdica (CASEIB 2010). • Pinzon-Morales, R.-D., Orozco-Gutierrez, A.-A. (2010). On The Wavelet Transform Customization via Machine Learning for Biological Signal Pro-. v.

(7) cessing. VII IEEE Latin-American Summer School on Computational Intelligence (EVIC 2010) • Pinzon-Morales, R.-D., Orozco-Gutierrez, A.-A., Castellanos, C.-G., CapobiancoGuido, R. (2011) The Discrete Lifting Shapelet Transform for Biological Pattern Recognition. The 5th International IEEE EMBS Neural Engineering Conference (NE2011). • Pinzon-Morales, R.-D., Orozco-Gutierrez, A.-A., Castellanos, C.-G., (2011) EEG Seizure Identification by using Optimized Wavelet Decomposition. In Engineering in Medicine and Biology Society (EMBC), 2011 Annual International Conference of the IEEE.. Journal Papers • Pinzon-Morales, R.-D., Garces-Arboleda M., Orozco-Gutierrez, A.-A. (2009). Automatic identification of various nuclei in the basal ganglia for Parkinson’s disease neurosurgery.. Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conference 2009, 3473– 3476. • Pinzon-Morales, R.-D., Orozco-Gutierrez, A.-A., Carmona-Villada, H. Castellanos, C.-G. (2010). Towards high accuracy classification of MER signals for target localization in Parkinson’s disease.. Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conference 2010, 4040 -4043. • Pinzon-Morales, R.-D., Orozco-Gutierrez, A.-A., Castellanos, C.-G. (2011) Novel signal-dependent filter bank method for identification of multiple basal ganglia nuclei in parkinsonian patients. Journal of Neural Engineering. vol. 8. num. 3, 036026, • Pinzon-Morales, R.-D., Orozco-Gutierrez, A.-A., Castellanos, C.-G, Guijarro, E, E. (2011) Wavelet Optimization for EEG Signal Classification. (In revision).. vi.

(8) Awards • 2nd Place at the CORAL Student Paper Competition of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) 2010. Towards high accuracy classification of MER signals for target localization in Parkinson’s disease. Buenos Aires, Argentina. • 2nd Place at the Student poster competition of the VII IEEE Latin-American Summer School on Computational Intelligence (EVIC 2010). On The Wavelet Transform Customization via Machine Learning for Biological Signal Processing. Santiago de Chile. Chile.. vii.

(9) Contents Publications. v. Awards. vii. Contents. viii. List of Figures. x. List of Tables. xiv. 1 Objectives 1.1 General Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Specific Objectives . . . . . . . . . . . . . . . . . . . . . . . . . .. 1 1 1. 2 Introduction. 2. 3 Biological Signals 3.1 Microelectrode Signals in Parkinson’s Disease . . . . . . . . . . . 3.2 Electromyographic Signals . . . . . . . . . . . . . . . . . . . . . . 3.3 Electroencephalographic Signals in Epilepsy . . . . . . . . . . . .. 4 4 5 6. 4 Methods Background 4.1 The Lifting Scheme . . . . . . . . . . . . 4.2 The Discrete Wavelet Transform . . . . . 4.3 The Discrete Wavelet Packet Transform . 4.4 The Discrete Shapelet Transform . . . . 4.5 Genetic Algorithms . . . . . . . . . . . .. viii. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. 8 8 10 11 11 15.

(10) CONTENTS. 5 Optimization Methodology 5.1 Preliminary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1.1 Input Database Generation . . . . . . . . . . . . . . . . . 5.1.2 Wavelet Analysis/Decomposition . . . . . . . . . . . . . . 5.1.3 Feature Selection/Extraction . . . . . . . . . . . . . . . . . 5.1.4 Fitness Function . . . . . . . . . . . . . . . . . . . . . . . 5.1.5 GA Operations . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Summarizing Wavelet and Shapelet Optimization: The SDFB and the DLST . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 17 17 17 18 19 20 23. 6 Experiments and Results 6.1 Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.1 Microelectrode Recordings (MER) . . . . . . . . . . . . . . 6.1.2 Electromyographic Signals (EMG) . . . . . . . . . . . . . . 6.1.3 Electroencephalographic signals (EEG) . . . . . . . . . . . 6.2 Identification of Basal Ganglia by Means of MER . . . . . . . . . 6.2.1 Wavelet Optimization for the SDFB . . . . . . . . . . . . . 6.2.2 Discrimination Potential of LFP, Spikes, and Raw Microelectrode Recordings . . . . . . . . . . . . . . . . . . . . . 6.2.3 Classification Performance . . . . . . . . . . . . . . . . . . 6.3 Identification of Hand Movements by Means of EMG . . . . . . . 6.3.1 Wavelet and Shapelet Optimization . . . . . . . . . . . . . 6.3.2 Classification Results . . . . . . . . . . . . . . . . . . . . . 6.4 Identification of EEG-Seizures . . . . . . . . . . . . . . . . . . . . 6.4.1 Wavelet and Shapelet Optimization . . . . . . . . . . . . . 6.4.2 Off-line Wavelet Packet Best Basis Selection . . . . . . . . 6.4.3 Classification Performance . . . . . . . . . . . . . . . . . .. 28 28 28 29 29 31 31. 7 Conclusions and Future Work 7.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Known Limitations and Extensions: Future Work . . . . . . . . .. 49 49 50. Bibliography. 53. ix. 23. 34 37 38 39 39 42 42 43 44.

(11) List of Figures 3.1. 3.2 4.1. 4.2. 4.3. 4.4 4.5 4.6. Typical microelectrode recording components. (Top) LFP component that corresponds to the low frequency band (< 150 Hz). (Middle) Spike component relating to the middle–high frequency ([500 − 10] kHz). (Bottom) a short–time microelectrode pattern of the TAL nucleus. . . . . . . . . . . . . . . . . . . . . . . . . . . . Typical EMG signal acquired from the forearm of a healthy patient. (Rigth) One-level decomposition with the lifting scheme (LS). The input signal x is split into odd xo and even samples xe by the lazy (1) wavelet. Then the sequence w2 is obtaining by predicting the odd samples from the even samples with the operator p. The sequence (1) w1 is obtained by updating the even samples with the operator u. (Left) Simplify diagram of the LS . . . . . . . . . . . . . . . . Three scale DWT decomposition and its associated time-frequency space. It is important to notice that the dyadic decomposition of the DWT produces a logarithmic fixed resolution on the timefrequency space. d stands for the detail coefficients while c stands for the approximation coefficients. Super index (l) indicates the decomposition level or scale. . . . . . . . . . . . . . . . . . . . . . DWT filter bank corresponding to a type I polyphasic factorization with a downsampling factor of 2, where He (z) and Ho (z), denote the even and odd tabs of the filter, respectively. . . . . . . . . . . Two different basis from the WPT decomposition tree and its respective time-frequency planes. . . . . . . . . . . . . . . . . . . . Wavelet Packet decomposition tree constructed by cascaded LS. . Typical flow chart for genetic algorithm based optimizations. . . .. x. 5 6. 8. 10. 11 12 12 15.

(12) LIST OF FIGURES. 5.1. 5.2. 5.3 5.4. 6.1 6.2 6.3. 6.4. Proposed methodology comprising four stages.(Red) Input database generation, (Blue) wavelet analysis/decomposition, (Orange) feature selection/extraction, (Green) fitness function and (Purple) GA operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . Flow chart of the GA–based optimization procedure for both shapelet and wavelet. Red path stands for the shapelet design through the fractal dimension metric. The blue path demarcates the computation of the fitness function regarding the wavelet design. . . . . . Flow chart of the GA–based optimization procedure for the signal– dependent filter bank . . . . . . . . . . . . . . . . . . . . . . . . . Flow chart of the GA–based optimization procedure for Discrete Lifting Shapelet Transform . . . . . . . . . . . . . . . . . . . . . .. 17. 24 25 26. Hand movements. 01 Closing, 02. Opening, 03.Flexion, 04. Extension, 05. Supination . . . . . . . . . . . . . . . . . . . . . . . 29 Examples of five different sets of EEG signals taken from different subjects. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 Time response of the associated wavelet and scaling function (dot line) of the customized operators (u and p) after the off-line optimization procedure. Operator values are: u = [0.0278 −0.2070 0.4292 0.4292 − 0.2070 0.0278], p = [0.0059 − 0.0488 0.2930 0.2930 − 0.0488 − 0.0059]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Low pass filter response associated to the operator u for different input database sizes on the optimization procedure. The Y-axis has been normalized. Current experiment is performed ten times. Outcomes are shown in terms of mean values and standard deviations (vertical lines). a) small-sized database. b) medium-sized database. c) large-sized database. . . . . . . . . . . . . . . . . . . 33. xi.

(13) LIST OF FIGURES. 6.5. Classification performance of the customized wavelet (blue line) and several existing wavelets (red line) including: Daubechies from order 2 to 7 (Db2, Db3, Db4, Db5, Db6, Db7), Symlet from order 3 to 7 (sym3, sym4, sym5, sym6, sym7), and Cohen–Daubechies– Feauveau of orders: cdf1.1, cdf1.3, cdf1.5, cdf2.2, cdf3.3. Notice that for the sake of visual comparison the standard deviation and classification performace of the customized wavelet is repetead on top of each classical wavelet. . . . . . . . . . . . . . . . . . . . . . 6.6 3D spaces obtained for (a) LFP, (b) spikes, and (c) raw microelectrode recording using the SDFB. The best combination of 3 (1) (2) features is the peak’s count of w2 , the curve length of w2 , and (3) the RMS value of w2 . Each class in the database is represented by a color. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.7 Classification performance under different noise-to-signal ratios. Signals are added with white Gaussian noise. . . . . . . . . . . . . 6.8 Frequency response of the shapelet-based wavelet functions designed for every class in the database. (Bold line) Low pass filter. (a) Closing, (b). Opening, (c).Flexion, (d). Extension, (e). Supination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.9 Frequency response of the Signal-dependent filter bank based wavelet function. (Bold line) Low pass filter . . . . . . . . . . . . . . . . . 6.10 Frequency response of the Daubechies wavelet or order 4. (Bold line) Low pass filter . . . . . . . . . . . . . . . . . . . . . . . . . . 6.11 Temporal response of the wavelet and shapelet functions associated to the optimized filters u and p, along with their frequency response. Results are given considering ten runs of the GA-based optimization with random initializations. (a) Wavelets functions, (b) Low-pass filter u ,(c) High-pass filter p associated to the SDFB optimization. (d) Shapelet functions, (e) and (f) Low-pass and High-pass response, respectively, belonging to the DLST . . . . . 6.12 Best basis of the wavelet packet binary decomposition tree. Details of the wavelet nodes, time resolution and associated frequency bands are given in Table 6.1 . . . . . . . . . . . . . . . . . . . . .. xii. 34. 35 38. 40 41 41. 43. 44.

(14) LIST OF FIGURES. 6.13 Number of PCA components against classification performance regarding the four cases under consideration. (Right) SDFB methodology. (Left)SDFB+WPD methodology6.3 . . . . . . . . . . . . . 6.14 3D feature spaces generated from the SDFB coefficients regarding Case IV including the fitness functions, (a) VDB ,(b) VDI , (c) Γ . .. xiii. 47 48.

(15) List of Tables 6.1 6.2. Optimized LS operators for different input database sizes . . . . . Sensitivity (E) and Specificity (F) of the LFP, Spike, and raw microelectrode recording 3D spaces generated with the SDFB . . . . 6.3 ROC and statistical analysis after employing the proposed SDFB approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 Daubechies Discrete Wavelet Transform of order 4 (%) . . . . . . 6.5 Signal-dependent filter bank(%) . . . . . . . . . . . . . . . . . . . 6.6 Discrete Shapelet Lifting Transform (%) . . . . . . . . . . . . . . 6.7 Best wavelet packet nodes and frequency bands . . . . . . . . . . 6.8 SDBF and DLST without post processing. SDFB stands for the signal-dependent filter bank including the VDB . On the other hand, the DLST, is the discrete lifting shapelet transform with fractal dimension working as metric . . . . . . . . . . . . . . . . . . . . . 6.9 Wavelet packet based decomposition without post processing. Bold values indicates were the performance was better that the presented in Table 6.8 with the dyadic decomposition . . . . . . . . . 6.10 Best results with post-processing. Bold values indicates were the performance was better that the presented in Table 6.8 and Table 6.9 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.11 Hubert’s and Dunn’s fitness function comparison in terms of classification performance. Bold values indicates performance that was better than DB index presented in Table 6.9 . . . . . . . . . . . .. xiv. 32 36 38 41 42 42 44. 45. 46. 46. 47.

(16) Chapter 1 Objectives 1.1. General Objective. To develop an evolutionary based optimization methodology for wavelet and shapelet functions that can be used for biological signal analysis.. 1.2. Specific Objectives. • To implement the wavelet and shapelet transform employing the lifting schemes, • To develop a genetic algorithm–based methodology for evolution of lifting schemes parameters involving the conditions of bi–orthogonality, lineal phase, compact support, and perfect reconstruction, • To incorporate measures of fitness into the evolutionary process that allow the construction of unique wavelet or shapelet functions from the biological signal’s information itself, • To validate the proposed methodology in several biological phenomena.. 1.

(17) Chapter 2 Introduction Wavelet analysis is a powerful tool for digital signal processing. It has been widely used in bioelectric signals including evoke-related potentials (ERP), electromyography signals (EMG), microelectrode recordings (MER), electrocardiogram (ECG), electroencephalograms (EEG), among others (Subasi [2007]; Wang et al. [2006]; Zhen et al. [2008]). Some of the principal advantages of wavelet transform are the compact support, and energy concentration. Basically, the wavelet transform is a convolution of the input signal with scaled and translated versions of one function called wavelet mother. Although there are plenty of wavelet prototypes in the literature, there is not an established rule that states which wavelet may be used for each application. Instead of that, it is a usual task for the researcher to test more o less arbitrarily different wavelet shapes in order to find one suited. A typical procedure for this job is to select a wavelet that resembles the shape of the interest signal; however, this is far from optimal. To solve the above issues several works have been proposed including LS with deterministic and evolutionary approaches. On one hand, analytical methods to adaptive synthesis have shown to be burdensome, owing to the complexity of mathematical conditions on orthonormality, symmetry, compactness, and smoothness (Arpaia et al. [2006]; Capobianco Guido et al. [2008]). On the other hand, evolutionary approaches, such as genetic algorithms (GAs), cultural algorithms (CAs), and ant systems (AS), have presented newsworthy features for wavelet design. For example, Zhen et al. (Zhen et al. [2008]) using an ensemble of GA and Lagrange Optimization is able to design a completely new wavelet shape from the analyzed signal. Arpaia et al. (Arpaia et al. [2006]) introduced addi-. 2.

(18) tional speed improvements to the GA with the AS, reporting better performance than existing wavelets for signal denoising. In this work the capabilities of the lifting schemes, which are a technique for both designing wavelets and performing the discrete wavelet transform and genetic algorithms are extended for wavelet synthesis in two scenarios: the signal– dependent filter bank and the discrete lifting shapelet transform. The former, is an evolutionary method based on the LS aimed at the synthesis of mother wavelets that exhibit unique time–frequency properties. Furthermore, the designed wavelet function is suppose to produce features spaces with maximum class separability due to the employment of clustering metrics as fitness functions into the GA procedure. The later, called the Discrete Lifting Shapelet Transform (DLST), considers the intrinsic information of the signal under analysis, such as the shape, or the frequency dynamic, to construct unique mother wavelet completely customized to the application. The DLST is inspired on two works: the Discrete Shapelet Transform and the Signal-Dependent Filter Banks. The former, described in Capobianco Guido et al. [2008], relates the analytical creation of new mother wavelets that provide an adequate framework to find the time-support of frequencies, and, at the same time, match patterns. The later corresponds to the construction of new mother wavelets based on the input signal itself, which exhibit unique time-frequency response, using lifting schemes and evolutionary techniques, as mentioned above. The original Shapelet procedure may be burdensome, owing to the complexity of mathematical conditions of orthogonality, symmetry, compactness, and smoothness. Current masters thesis is organized as follows: In chapter 3 a brief description about the biological phenomena that are being considered are rendered. Then, in chapter 4 the baseline of the employed techniques are given including the wavelet analysis and the genetic algorithms. In chapter 5 the proposed methodology for optimization of wavelets and shapelet is stated in detail. Next in chapter 6 the validation of the proposed methodology is carried out in three scenarios, firstly regarding the identification of basal ganglia by means of microelectrode recording analysis. Secondly, at the recognition of hand movement using electromyographic signals. Finally, at classification of epileptic seizures employing EEG signals. In the last part of this document conclusions and future work are stated.. 3.

(19) Chapter 3 Biological Signals 3.1. Microelectrode Signals in Parkinson’s Disease. Deep brain Stimulation (DBS) of the basal ganglia nuclei has become a standard treatment for advanced cases of Parkinson’s disease. Among the different nuclei within the basal ganglia, the Subthalamic nucleus (STN) is the preferred target, although other zones are also feasible (Gemmar et al. [2008]). Success in DBS procedures relies on the correct localization of the target nucleus, where the stimulation microelectrode is to be placed (Wong et al. [2009]). For this purpose, stereotactic frames and T1–weighted magnetic resonance images are used for preoperative planning of the path to the target. Additionally, electrophysiological techniques that can provide intraoperative information, such as microelectrode recordings (MER), are frequently used to refine the precision of targeting (Zaidel et al. [2009]). Microelectrode recordings are strong non–stationary signals transduced from the electrical activity generated by neurons. This activity is highly dependent on numerous aspects with nonlinear structure (Lim et al. [2010]). For example, fatigue phenomena that cause non–symmetrical behavior and systematic amplitude reduction of action potentials. Other external factors are the cortical pulse generated by respiratory and cardiac activity, microelectrode movements, and background noise. Microelectrode signals are mainly comprised of three electrical phenomena: spikes, local field potentials (LFP), and background electrical noise. Spikes stand for the superposition of action potentials produced. 4.

(20) 3. Biological Signals. Figure 3.1: Typical microelectrode recording components. (Top) LFP component that corresponds to the low frequency band (< 150 Hz). (Middle) Spike component relating to the middle–high frequency ([500 − 10] kHz). (Bottom) a short–time microelectrode pattern of the TAL nucleus. by short inward and outward transmembrane currents in most nearby neurons. The LFP represent the synaptic activity, whereas the electrical noise comes from several sources including remotely located units, measurement noise, and various artifacts (Kostis et al. [2009]). A typical MER signal is depicted in Fig. 3.1. In practice, skilled specialists use visual and acoustic clues of the microelectrode recording for determining the position of the target. However, elevated uncertainty and mistakes can arise from this highly subjective and human– dependent approach. Furthermore, microelectrode recordings from several nuclei may express similar activity, making the recognition task more arduous. In this sense, a computer–assisted system for automatic identification of nuclei would be of outstanding help during DBS implantation.. 3.2. Electromyographic Signals. Surface Electromyographic Signal (SEMG) processing has become an important tool in fields like biological pattern recognition, prosthetic control, clinical di5.

(21) 3. Biological Signals. agnosis and rehabilitation. These signals can be monitored noninvasively using electrodes attached to human skin. They represent the neuromuscular activity from the temporal and spatial aggregation of independent Motor Unit Action Potentials (MUAPs) in the muscle (Andrade et al. [2008]). Similar to other biological signals, SEMG signals have non-stationary behavior (Andrade et al. [2008]) due to: metabolic accumulations that produces muscular fatigue, changes in the impedance of the skin-electrode interface, or changes of the MUAP over the time or others conditions. In this sense, assiduous efforts have been made in recent years to improve classification accuracy of SEMG signals and to reduce computational complexity of this kind of recognition systems, making them suitable for implementation in real-time applications (Chu et al. [2005]; Huang and Chen [1999]; Marshall and Murphy [2008]). A typical EMG signal is depicted in Fig. 3.2.. 3.3. Electroencephalographic Signals in Epilepsy. Epilepsy is one of the most common neurological disorders with a prevalence of 0.6–0.8% of the world’s population. Two-thirds of the patients achieve sufficient seizure control from anticonvulsive medication, and another 8–10% could benefit from resective surgery. For the remaining 25% of patients, no sufficient treatment. Figure 3.2: Typical EMG signal acquired from the forearm of a healthy patient.. 6.

(22) 3. Biological Signals. is currently available (Tzallas et al. [2007]). The epilepsy is characterized by a sudden and recurrent malfunction of the brain, which is termed seizure. Epileptic seizures reflect the clinical signs of an excessive and hypersynchronous activity of neurons in the brain. Depending on the extent of the involvement of other brain areas during the course of the seizure, epilepsies can be divided into two main classes. Generalized seizures involve almost the entire brain, while focal (or partial) seizures originate from a circumscribed region of the brain (epileptic focus) and remain restricted to this region. Epileptic seizures may be accompanied by impairment or loss of consciousness: psychic, autonomic or sensory symptoms, or motor phenomena. Traditionally, suspected seizures are evaluated using a routine electroencephalogram (EEG), which is typically a 20-minute recording of the patients brain waves. Because a routine EEG is of short duration, it is unlikely that actual events are recorded. Routine EEGs may record interictal hallmarks of epilepsy, including spikes, sharp waves, or spike-and-wave complexes. However, diagnostic difficulties arise when a person has a suspected seizure, or a neurological event of unclear etiology, not obvious in the routine EEG. The current gold standard is the continuous EEG recording along with video monitoring of the patient, which usually requires patient admission. This is a costly endeavour, which is not always available.. 7.

(23) Chapter 4 Methods Background 4.1. The Lifting Scheme. The lifting scheme is a technique for both designing wavelets and performing the discrete wavelet transform. Actually it is worthwhile to merge these steps and design the wavelet filters while performing the wavelet transform (Sweldens [1996]). One-level decomposition with the LS is shown in Fig. 4.1, which is built by applying the polyphase filter decomposition over the discrete wavelet transform. This technique is regarded as a fast and efficient implementation of the wavelet transform. Moreover, the proper selection of the LS operators allows the construction of any orthogonal or bi–orthogonal wavelet.. Figure 4.1: (Rigth) One-level decomposition with the lifting scheme (LS). The input signal x is split into odd xo and even samples xe by the lazy wavelet. (1) Then the sequence w2 is obtaining by predicting the odd samples from the even (1) samples with the operator p. The sequence w1 is obtained by updating the even samples with the operator u. (Left) Simplify diagram of the LS. 8.

(24) 4. Methods Background. During the LS decomposition the following three steps are involved: division, prediction, and update. In the division step, the input signal x = {x[n] : n = 1, . . . , N } is split into even samples xe = {x[2n] : n = 1, . . . , N }, as well as into odd samples xo = {x[2n − 1] : n = 1, . . . , N }. This procedure is also referred as lazy wavelet. Then in the prediction step, the operator p is applied on xe (l−1) to predict xo (l−1) with the main purpose of eliminating low order polynomials (l) (l) from x, and thus, obtaining detail coefficient sequence w2 = {w2 [n] : n = 1, . . . , N/(2l )}, which is described as follow: X. Np /2 (l) w2 [n]. =. [n] x(l−1) o. −. [n + r], p[r]x(l−1) e. r=−Np /2+1. where p = {p[r] : r = 1, . . . , Np } are coefficients of the prediction operator, super index (l) indicates the decomposition level, where l = 0 is the original signal x, and Np is the support of p. Next, in the update stage, an update on the even samples xe (l−1) is accom(l) plished by using the update operator u on the detail coefficients w2 , and adding (l) (l) the result to xe (l−1) . The update sequence w1 = {w1 [n] : n = 1, . . . , N/(2l )} can be considered a rough view of x, that is, X. Nu /2−1 (l) w1 [n]. =. x(l−1) [n] e. −. (l). uj w2 [n + j − 1],. j=−Nu /2. where u = {u[r] : r = 1, . . . , Nu } are values of the update operator, and Nu is the support of u. Some properties of the lifting scheme are: • Each transform by the lifting scheme can be inverted so that perfect reconstruction is secured. • Conditions of symmetry, linear phase, filter normalization, and compact support are secured.. 9.

(25) 4. Methods Background. Figure 4.2: Three scale DWT decomposition and its associated time-frequency space. It is important to notice that the dyadic decomposition of the DWT produces a logarithmic fixed resolution on the time-frequency space. d stands for the detail coefficients while c stands for the approximation coefficients. Super index (l) indicates the decomposition level or scale.. 4.2. The Discrete Wavelet Transform. The one-dimensional Discrete Wavelet Transform (DWT) represents a real valued discrete time signal in terms of shifts and dilations of a lowpass scaling function and a bandpass wavelet function. The DWT decomposition is multiscale: it consists of a set of approximation coefficients c(0) [n] which represent coarse signal information at scale j = 0 and a set of detail coefficients d(j) , which represent detail information at scales j = 1, . . . , J. Furthermore, the DWT provides a timefrequency representation of the signal by employing the mention dyadic filter bank decomposition with logarithmic resolution as shown in Fig. 4.2. Any biorthogonal wavelet transform can be represented as a perfect reconstruction multirate filter bank by splitting each wavelet filter into its type one polyphase components (Claypoole et al. [1998]; Vaidyanathan [1993]). In the case of the DWT the simplification can be implemented as shown in Fig. 4.3. Considering that that H(z) = He (z) + z −1 Ho (z) and the noble identity that states that filtering a signal with He (z) and then downsampling by two is equivalent to downsampling the signal by two and then applying He (z 2 ) (Vaidyanathan [1993]). Further factorization of the DWT representation yields the relation between the. 10.

(26) 4. Methods Background. Figure 4.3: DWT filter bank corresponding to a type I polyphasic factorization with a downsampling factor of 2, where He (z) and Ho (z), denote the even and odd tabs of the filter, respectively. LS and the DWT as follows: He (z) = 1 − P (z)U (z), Ho (z) = U (z) Ge (z) = −P (z),. (4.1). Go (z) = 1. (4.2). H(z) = 1 − P (z 2 )U (z 2 ) + z −1 U (z 2 ),. (4.3). G(z) = −P (z 2 ) + z −1. (4.4). or equivalently,. with similar expressions for Ĥ(z) and Ĝ(z).. 4.3. The Discrete Wavelet Packet Transform. Classically the Wavelet Packet Transform (WTP) is constructed by taking the wavelet transform of the detail and approximation coefficients simultaneously in the DWT. The main advantage of this process is that the resolution of the time-frequency plane derived from the analysis is completely customized by the selection of different basis of decomposition. For example, in Fig. 4.4 are shown two different basis of the WPT and its respective time-frequency planes.. 4.4. The Discrete Shapelet Transform. The Discrete Shapelet transform (DST), first introduced by Guido et al. (Capobianco Guido et al. [2008]), is a novel transform used to find the time-support of frequencies, and, at the same time, match patterns. The elements that form. 11.

(27) 4. Methods Background. Figure 4.4: Two different basis from the WPT decomposition tree and its respective time-frequency planes.. Figure 4.5: Wavelet Packet decomposition tree constructed by cascaded LS. the Shapelet structure, which are similar to the elements of the quadrature filter bank in the DWT structure, are (Capobianco Guido et al. [2008]).. 12.

(28) 4. Methods Background. • a low-pass half-band finite impulse response filter (FIR), g[n], which has a frequency response G[ω] so that G[ω = π] = 0, and a phase response that is not necessarily linear, • the corresponding FIR high-pass mirror filter, h[n], defined as h[n] = (−1)n h[N − n − 1] due to perfect reconstruction, • the support-size of the filters, i.e., the number of coefficients or the length, N , (N < 4), that must be even, • the scaling and wavelet functions, as discussed later. The requirements above for analysis filters go [n] and he [n] along with N , force the filters to form a perfect-reconstruction filter bank (PRFB) (Vaidyanathan [1993]), i.e., the anti-aliasing conditions, in Eq. 4.5, and the no-distortion condition, in Eq. 4.6, both in z-domain, are satisfied. H̄[z] = Q[z], Q̄[z] = −P [−z]. (4.5). P̄ [z]P [z] + Q̄[z]Q[z] = 2z −N +1. (4.6). Once the coefficients of one of the FIR filters specified above are determined, Shapelet can be completely defined. To determine them, the next algorithm, with five sequences of steps, has to be followed: sequence A, sequence B, sequence C, sequence D (D1 and D2) and sequence E. Sequences A, B, C, and D can happen in parallel, but E requires the completion of the previous sequences to be carried out. A detailed description is presented next (Capobianco Guido et al. [2008]). • Step A: Let the filter have unitary energy, i.e., N −1 X. g[k]2 = 1. (4.7). k=0. • Step B: Impose the orthogonality conditions among the translations of the filter: N −1 X g[k]g[k + 2l] = δ0,l (4.8) k=0. where δ is the Dirac delta, and l ∈ Z.. 13.

(29) 4. Methods Background. • Step C: Produce a system with N/2 − 1 vanishing moments, i.e., N −1 X. g[k]k b = 0. (4.9). k=0. for b = 0, 1, . . . , N/2 − 2. • Step D1: Calculate the fractal dimension, Dm , (1 > Dm > 2) of the nonconstant matching signal m[n], i.e., the signal that Shapelet is intended to match, which is of unconstrained length, by using the power spectrum method, as explained later and introduces in (Marwan [2004]). • Step D2: Calculate the first level DST of m[n], (DST(m[n])), using the generic filters h[n] and g[n] of support N in the unknowns h[0], . . . , h[N 1] and g[0], . . . , g[N 1]. Then, obtain the fractal dimension, Dt , of DST(m[n]), that is a non-linear equation in the same unknowns, and let Dt = Dm. (4.10). aiming at maintaining the degree of self-similarity between the original and transformed signals. The equation h[k] = (−1)k+1 g[N − k − 1]. (4.11). must be used so that Dt becomes a function of g[n] only, • Step E1: Group together the only equation of step A, the N/21 equations of step B, the N/21 equations of step C, and the only equation of step D2, resulting in a non-linear system of N equations in N unknowns. • Step E2:Solve the system using any iterative numerical procedure, and output the set of coefficients g. Regarding the fractal dimension its computation follows the procedure described in (Marwan [2004]), refereed as power spectral method. Given the digital power spectrum Si ≡ S(ki ) ∈ R1×Ns having been obtained by applying an fast Fourier transform to the digital fractal signal (or alphabet), the fractal dimension is computed as F = ((5 − β)/2), where β is derived from the least square. 14.

(30) 4. Methods Background. approximation shown next PNs PNs P s Ns ( N i=0 ln(Si ) ln(|ki |)) − ( i=0 ln(|ki |))( i=0 ln(Si )) β= PNs P N /2 s ( i=0 ln(|ki |))2 − Ns ( i=2 ln(|ki |)2 ) where Si is suppose to include only the half-space of the spectrum without the DC value, since the power power spectrum of a real signal is symmetric.. 4.5. Genetic Algorithms. Genetic algorithm is one of the most widely used artificial intelligent techniques belonging to the area of evolutionary computation. Genetic algorithm based on the mechanisms of natural selection and genetics, has been developed since 1975 (Mitchell [1996]) and has been applied to a variety of optimization and search problems (Arpaia et al. [2006]; Kumsawat [2010]; Zhen et al. [2008]). GA has been proven to be very efficient and stable in searching for global optimum solutions. Usually, a simple GA is mainly composed of three operations: selection, genetic operation, and replacement. A brief summary for implementing GA can be stated as follows (Kumsawat [2010]). Defining the solution representation of the system is the first task of applying GA. GA uses a population, which is composed of a group of chromosomes, to represent the solutions of the system. The solution in the problem domain can then be encoded into the chromosome in the GA domain and vice versa. Initially, a population is randomly generated.. Figure 4.6: Typical flow chart for genetic algorithm based optimizations.. 15.

(31) 4. Methods Background. The fitness function then uses objective values from objective function to evaluate the fitness of each chromosome. The fitter chromosome has the greater chance to survive during the evolution process. The objective function is problem specific; its objective value can represent the system performance index (e.g., an error). Next, a particular group of chromosomes is chosen from the population to be parents. The offspring is then generated from these parents by using genetic operations, which normally are crossover and mutation. Similar to their parents, the fitness of the offspring is evaluated and used in replacement processes in order to replace the chromosomes in the current population by the selected offspring. The GA cycle is then repeated until a desired termination criterion is satisfied, for example, the maximum number of generations is reached or the objective value is below the threshold. The GA cycle is shown in Fig. 4.6 where the phenotype is the coding scheme used to represent the chromosomes.. 16.

(32) Chapter 5 Optimization Methodology 5.1. Preliminary. The proposed methodology for wavelets and shapelets synthesis is depicted in Fig. 5.1. It comprises five main stages represented by colors. (i) Input database generation, (ii) wavelet analysis/decomposition, (iii) feature selection/extraction, (iv) fitness function and (v) GA operations.. 5.1.1. Input Database Generation. The first stage, refereed as input database generation, is a major step in the proposed methodology. It is responsible for feeding the next stages with repre-. Figure 5.1: Proposed methodology comprising four stages.(Red) Input database generation, (Blue) wavelet analysis/decomposition, (Orange) feature selection/extraction, (Green) fitness function and (Purple) GA operations. 17.

(33) sentative information from the biological phenomena, including the different types of dynamics within it, with the purposed of evolve the LS operators in terms of such information. Indeed, the operators are generated from the phenomena itself. Considering a initial database Λ ∈ RN ×M , where M is the number of observations of the phenomena, containing c different types of dynamics (classes) each one of N samples so that Λ = {λ1 , λ2 , . . . , λc }, where λc ∈ RN ×Mc , and Mc is the number of patterns per class. The selection of a sample database Ω ⊆ Λ, should provide information from each class, i.e., Ω = {ω1 , ω2 , . . . , ωc }, with, ωc ∈ RN ×M̄c , M̄c < Mc . The procedure for selecting the input set of signals Ω, may follow the 2way cross validation strategy, where the database is split into the training and validation groups. The construction of mentioned groups is performed by random selection seeking an uniform distribution of observations per classes. For instance, the Ω may be constructed by taking 30% of the samples of each class within the initial database Λ.. 5.1.2. Wavelet Analysis/Decomposition. In the present stage, the main purpose is to generate a set of wavelet coefficients for each pattern in the input database. Three points are to be taken into account. First, the scale level, second the decomposition tree and third the LS constrains. Regarding the scale level it should be selected accordingly to the frequency bands of interest. In the case of the decomposition three, it is recommended in this work to choose the dyadic over the binary scheme. The main reason is that the later may introduce undesired information due to the redundancy nodes. However, the binary process could also exhibit a more flexible time-frequency representation of the signal by using an adequate basis selection algorithm. This remains as a open issue. Regarding the LS constrains, as expressed in Zhen et al. [2008], each one of the operators, p and u, can lead to its own wavelet function with unique time–frequency features. In this sense, the following constrains are introduced to ensure that the associated wavelet function is suitable for linear phase, compact support, and perfect reconstruction. On one hand, the follow symmetrical linear. 18.

(34) phase constraints are imposed (Gouze et al. [2004]): p[r] = p[−r + 1],. r = 1, 2, ..., Np /2,. (5.1). u[j] = u[−j + 1],. j = 1, 2, ..., Nu /2,. (5.2). On the other hand, the filtering normalization conditions: XNp /2 j=1. XNu /2 j=1. p[r] = 1/2,. (5.3). u[j] = 1/4.. (5.4). The off-line optimization procedure under the given constraints Eq. (5.1, 5.2, 5.3, 5.4) is presented in the following sections.. 5.1.3. Feature Selection/Extraction. Once the wavelet coefficients w are generated, it is recommended to reduce its dimension by extracting a set of metrics Φ, which may have more physical meaning than the coefficients themselves. For instance, it has been exposed in (Wong et al. [2009]) that the following morphological metrics are discriminant for MER signals. N −2 1X Φ1 = max{0, sgn(x[n + 2] − x[n + 1]) − sgn(x[n + 1] − x[n])}, 2 n=1. Φ2 =. N −1 X. |x[n + 1] − x[n]|,. (5.5). (5.6). n=1. s Φ3 =. PN. x[n]2 , N. n=1. (5.7). where Φ1 is the peak’s count in x, Φ2 is the curve length, and Φ3 is the root mean square amplitude of x. Other typical metrics include: Φ4 = max(|x|), maximum value of the absolutes,. 19.

(35) Φ5 = Φ6 =. 1X 2 x , energy, 2 N. 1 X |x|, normalized mean, N N. p γ(M M T ), square root of the eigenvalues of, M , X Φ8 = − x[n]2 log(x[n]2 ), Shannon’s entropy,. Φ7 =. n∈N. 1 X |x[n]|, sum of the absolutes, N n∈N P 1 (x[n]−x)3 N n∈N = 3/2 , kurtosis, P 1 (x[n]−x)2 N. Φ9 =. Φ10. n∈N. 1 N. Φ11 = 1 N. P. (x[n]−x)4 n∈N 2 − 3, skewness, P 2 (x[n]−x) n∈N. where matrix M ∈ R2×N/2 is constructed by indexing all the decomposition coefficients of x on a zero matrix, and x is the mean value.. 5.1.4. Fitness Function. The fitness function can be regarded as a major step in the synthesis of wavelet functions. The selection of such function should be taken accordingly to the application under consideration. For example, if the application is signal compression then the desire fitness function should be an informative metric, such as Shannon’s entropy so that the optimized wavelet will produce representations with maximum information. Another case, named, pattern classification, would required a different function. Thus, the objective is now to generate a wavelet function that aims the classification procedure. The current manuscript deals with the later case. Several cluster validation measures are being considered in order to design wavelets and shapelets that allow the construction of feature spaces with maximum class separability. The cluster measures considered include the Modified Hubert Statistic (MH) (Sledge. 20.

(36) et al. [2010]), the Davies-Bouldin index (DBI) (Davies and Bouldin [1979]), and the generalized Dunn’s index (DI) (Bezdek and Pal [1998]). Let X = {x1 , x2 , . . . , xn } ∈ Rp a set of n features in p-space. Given a crisp set of labels for every element in X, say O = o1 , o2 , ..., oc belonging to c classes, it can be arrayed as the columns of a 1 × c ∗ n partition matrix U (X) = U = [uk ]. The value uk indicates the membership of the xk . Modified Huberts statistic (MH): The original Hubert’s Γ statistic (Sledge et al. [2010]) assesses the fit between the data and any crisp structure imposed on it by U . Let R = [rij ] be a n × c ∗ n proximity matrix; is the observed proximity between objects i and j given by the norm rij = kxi xj k, although any other norm can be employed. Q = [qij ] is an n × n matrix defined in terms of the hard c-partition U of X ( |Q(U )|ij = qij =. 0 uj = ui ∃classc 1 otherwise. ) ,. (5.8). Hubert’s Γ statistic is the point serial correlation coefficient between any two matrices. When the two matrices are symmetric, Γ can be defined in its raw form as n−1 X X Γ(R, Q(U )) = rij qij (5.9) i=1 j=i+1. In its normalized form, Γ becomes the sample correlation coefficient between the entries of R and Q n n−1 1 X X Γ̄(R, Q(U )) = (rij − r̄)(qij − q̄)/(sR sQ ) M i=1 j=i+1. (5.10). where M = n(n − 1)/2 is th total number of entries under the double summation, r̄ and q̄ are the mean values of the proximity and Q matrix (∀i = 1, . . . , n, j = i+1, . . . , n). sr and sq are the standard deviations, or dispersion, of the proximity rij and qij values. For the normalized index −1 < Γ < 1. Above definition of the Hubert’s statistics would required the computation of all !n possible permutations and then finding its histogram. Indeed, computationally prohibitive. The modified Hubert’s statistic (MH) abandons the goodness of. 21.

(37) fit strategy, and replaces it with a geometric method that is based on intuitively natural principles (Sledge et al. [2010]). Let rewrite the definition for Q(U ) in terms of the Euclidean distance between clusters centers V̄ (U ) vi and vj , as follows Q(U, V̄ (U )) = [kvL(i) − vL(j) k2 ], i, j = 1, 2, . . . , n. (5.11). where L(i) = c if the ith sample is in the cth cluster. Using Eq. 5.11 in Eq. 5.9 and Eq. 5.10 yields, Γ(R, Q(U, V̄ (U )))Hubert’s modified raw statistic. (5.12). Γ̄(R, Q(U, V̄ (U )))Hubert’s modified normalized statistic. (5.13). Values of Γ̄ near to 1 means that the hard partition U imposed in X is a good fit. Indeed, the data clusters are far form each other and condensed. The contrary happens near to 0. Davies-Bouldin Index (DBI): The DBI index is a function of the ratio of the sum of within-cluster scatter to between-cluster separation (Davies and Bouldin [1979]), and like Hubert’s measure, it also uses both the clusters and their sample means V̄ (U ). Since scatter matrices depend on the geometry of the clusters. Indeed, the DB index has both a statistical and geometric rationale. Define the within th cluster scatter and the between th and th cluster distance as ! 1 X kx − vi k2 (5.14) Si = |Xi | x∈X i. and, di,j =. ( p X. )1/2 |vsi − vsj |2. (5.15). s=1. Si,j is the square root of the second moment of the points in cluster i with respect to their mean, and is a measure of dispersion of the points in cluster i. di,j is the Minkowski distance of order between the centroids which characterize clusters i and j. Then, define Si (U ) + Sj (U ) Ri = max (5.16) i,j6=i di,j (U ) 22.

(38) 5. Optimization Methodology. Now the DaviesBouldin (DB) index is the worts case of separation that is expressed as: c 1X VDB (U, V̄ (U ))) = Ri (U ) (5.17) c i=1 It is shown that lower values of the DB index indicate a higher degree of cluster separability and compactness. Generalized Dunn’s index (DI): Dunns index (DI) is based on geometrical considerations that have the same basic rationale as the DBI in that they are both designed to identify sets of clusters that are compact and well separated (Bezdek and Pal [1998]).Given to subsets H and T of Rp along with a fixed δd. The standard definitions of the diameter ∆ of H and the set distance δd between H and T are ∆(H) = max {d(x, y)} (5.18) x,y∈H. and, δ(H, T ) =. min {d(x, y)}. x∈H,y∈T. (5.19). Given any crisp partition U (X), Dunn specified the separation index of U (Xc )∀c as δ(U, X) VDI (U ; X) = min min (5.20) 1≤h≤c 1≤t≤c,t6=s max1≤c {∆(Xk )}. 5.1.5. GA Operations. Once the fitness function is calculated aimed at the application goal, i.e, classification, it is used to evaluate the fitness of each chromosome and then perform Ga operations that are analogous to those which occur in the natural world: survival of the fittest, or selection; reproduction (crossover, also called recombination); and mutation. Refereed to Sec. §4.5. 5.2. Summarizing Wavelet and Shapelet Optimization: The SDFB and the DLST. The optimization procedure for the wavelet function implied in the signal-dependent filter bank (SDFB), and the shapelet function involved in the discrete lifting Shapelet transform (DLST) is depicted in Fig. 5.2, as expressed in previous. 23.

(39) 5. Optimization Methodology. Figure 5.2: Flow chart of the GA–based optimization procedure for both shapelet and wavelet. Red path stands for the shapelet design through the fractal dimension metric. The blue path demarcates the computation of the fitness function regarding the wavelet design. section. The common steps (shown in purple color) include the input database selection, Sec. §5.1.1, the wavelet decomposition with LS, Sec. §5.1.2, and the GA operations, Sec. §5.1.5. Additional details about the particular steps for the SDFB and the DLST are given next. In the case of the SDFB, in the multi-class problem, the procedure can be resumed into the following steps (see Fig. 5.3): • Step A: Initialize N/2 − 1 values of the operators, u and p, of the LS, with random values inside the interval [−1 − 1], where N is the desire order of the filters. The remain N/2 + 2 values are calculated using the linear and normalization constrains Eq. 5.1, Eq. 5.2, Eq. 5.3, and Eq. 5.4 • Step B: Extract the training subset Γ, including n patterns as described in Sec. §5.1.1. • Step C1: Decompose the n patterns with the LS up to level l, and extract n feature vectors Φ, using any of the metrics presented in Sec. §5.1.3. • Step C2: Compute the fitness function employing any of the cluster validation measures in Sec. §5.1.4. 24.

(40) 5. Optimization Methodology. Figure 5.3: Flow chart of the GA–based optimization procedure for the signal– dependent filter bank • Step C3: Perform GA operations. The outcome of the above procedure is a pair of LS operators,u and p, that maximizes the distance inter class and minimizes the intra class dispersion of the input database, Λ, in the d-dimensional feater space, Φ. The sequence involved in the construction of the DLST involving the shapelet design, presented in the flow chart Fig. 5.4, is as follows: • Step A: Calculate the fractal dimension, Dm of the non-constant matching signal m[n], i.e., the signal that Shapelet is intended to match, which is of unconstrained length, by using the power spectrum method, as explained in Sec. §4.4. Other methods were discarded due to their higher order of complexity and/or similar results worsen as the fractal dimension increases. 25.

(41) 5. Optimization Methodology. Figure 5.4: Flow chart of the GA–based optimization procedure for Discrete Lifting Shapelet Transform • Step B1: Initialize N/2 − 1 values of the operators, u and p, of the LS, with random values inside the interval [−1 − 1], where N is the desire order of the filters. The remain N/2 + 2 values are calculated using the linear and normalization constrains, Eq. 5.1, Eq. 5.2, Eq. 5.3, and Eq. 5.4 (l). • Step B2: Calculate the approximation coefficients w2 of level l by decomposing the input signal m[n]. • Step B3: Obtained the shapelet lifting fractal dimension Dls =DLST(m[n]) from the approximation coefficients. • Step B4: Compute the absolute error between the target fractal dimension 26.

(42) 5. Optimization Methodology. and the wavelet fractal dimension, e = |Dm − Dls |. • Step C: Perform Genetic Algorithm’s operations, with minimization fitness function e, and repeat from B1 to C until reached the given number of generations. For the particular case that the m[n] is the input signal, and both, u and p, are the shapelet operators optimized from m[n], the fractal dimensions of m[n] and DLST(m[n]) are the same, i.e., the DLST preserves the degree of self similarity of the input signal for it was designed.. 27.

(43) Chapter 6 Experiments and Results In the present section three biological phenomena are considered to test the proposed methodology. Firstly, the performance of the wavelet optimization is assessed withing the basal ganglia nuclei identification scenario. Then, wavelet and shapelet procedure are compared in the recognition of hand movements form EMG signals. Thirdly, the EEG problem is considered along with both the shapelet and wavelet methodologies including different fitness functions. Additionally, computational performance, visualized 3D plots and KDC based measures are presented.. 6.1 6.1.1. Databases Microelectrode Recordings (MER). Intraoperative acquisitions were made on unmedicated awake patients that underwent DBS implantation. Surgeries that were carried out in the University General Hospital of Valencia were labeled by both specialists in neurophysiology and electrophysiology, according to the affected brain zone. The equipment used in the acquisition was the LeadPointTM Medtronic (Medtronics Functional Diagnostics). The sampling frequency was 24 kHz and 16-bit resolution and lasted 1 s. In total, there are 216 recordings coming from the following brain zones: 66 signals from the Thalamus (TAL), 25 signals – the Subthalamic nucleus (STN), 38 signals – the Substantia Nigra pars Reticulate (SubNR), and 87 signals from the Zone Incerta (ZI). Here in, this database is refereed as UPV–DB.. 28.

(44) Figure 6.1: Hand movements. 01 Closing, 02. Opening, 03.Flexion, 04. Extension, 05. Supination. 6.1.2. Electromyographic Signals (EMG). Five hand movements were considered and labeled as follows: closing (CLO), opening (OPE), flexion (FLE), extension (EXT), and supination (SUP) (see Fig. 6.1). In total there are 500 SEMG signals acquired from the forearm of a nonamputee volunteer using the Delsys Bagnoli-4 system coupled with one parallelbar active SEMG sensor. For data acquisition, 16-bit per channel resolution, 2 kHz sampling frequency and 1000x voltage gain were used. The database was split into two groups using the k -fold cross validation methodology: 70% of the signals is used for the training and the 30% left is used for validation. Ten folds (k = 10) are generated randomly.. 6.1.3. Electroencephalographic signals (EEG). A public EEG dataset, which is available and described in (Andrzejak et al. [2001]) and includes recordings for both healthy and epileptic subjects, is used. In this section, a short description is given. Refer to Andrzejak et al. [2001] for further details. The complete data set embraces five sets (denoted AE) each containing 100 single-channel EEG segments, each one having 23.6 s duration. These segments were selected and cut out from continuous multi-channel EEG recordings after visual inspection for artifacts, e.g., due to muscle activity or eye movements. Sets A and B consisted of segments taken from five healthy volunteers with eyes open and closed, respectively, using surface EEG electrodes placed in a standardized (10-20) reference scheme. Sets C, D, and E are originated from EEG archive of presurgical diagnosis. Recordings form five patients were 29.

(45) selected, all of whom had achieved complete seizure control after resection of one of the hippocampal formations, which was therefore correctly diagnosed to be the epileptogenic zone. Type D recordings were acquired within the epileptogenic zone, while the type C patterns from the hippocampal formation of the opposite hemisphere of the brain. Sets C and D contained only activity measured during seizure free intervals, on the contrary the set E only contained seizure activity. The data were digitized at 173.61 samples per second using 12 bit resolution and they have the spectral bandwidth of the acquisition system, which varies from 0.5Hz to 85 Hz. Typical EEG are depicted in Fig. 6.2. In the present study, the above-described dataset is employed to create four different medical classification problems that are fed into the considered methodologies. • Case I: in the first , normal and seizures recordings are classified. The normal class includes only the A-type EEG segments while the seizure class includes the E-type. • Case II: In the second, all the EEG segments from the dataset were used and they were classified into two different classes: A, B, C, and D types. A−Type. 200 0 −200 0 500 0 −500 0 200 0 −200 0 1000 0 −1000 0 2000 0 −2000 0. 10. 15. 20. 25. 5. 10. 15. 20. 25. 5. 10. 15. 20. 25. 5. 10. 15. 20. 25. 5. 10. 15. 20. 25. E−Type. D−Type. C−Type. B−Type. 5. Time (s). Figure 6.2: Examples of five different sets of EEG signals taken from different subjects.. 30.

(46) are included in the first class and type E in the second class. This is also close to real medical applications, being slightly simpler than the previous, classifying the EEG segments into non-seizures and seizures. • Case III: The third is a three-class problem, that is, normal, seizure-free and seizure, but not all the EEG segments from the dataset were employed. The normal class includes only the A-type EEG segments, the seizure-free class the D-type EEG segments, and the seizure class the E-type. • Case IV: In the last, all the EEG segments from the dataset are used and they are classified into three different classes: A and B types of EEG segments were combined to a single class, C and D types were also combined to a single class, and type E was the third class. Case IV is the one closest to real medical applications including three categories; normal (i.e., types A and B), seizure-free (i.e., types C and D) and seizure (i.e., type E).. 6.2. Identification of Basal Ganglia by Means of MER. Testing of the proposed methodology, regarding the signal–dependent filter bank, embracing three experiments, is presented here. Firstly, the tuning of the SDFB is performed involving the customized wavelet function (Sec. §5), which is compared along with several baseline standard wavelets. Comparison is made in terms of classification performance. Secondly, the discrimination ability of LFP, spikes, and raw microelectrode recordings employing the SDFB is assessed. Lastly, white Gaussian noise is added to evaluate the behavior of the SDFB under noisy inputs. Receiver Operating Characteristic (ROC) analysis and statistical values are provided.. 6.2.1. Wavelet Optimization for the SDFB. The tuning of the SDFB relies on the GA based optimization of both operators, p and u, during the LS decomposition. Before doing so, a training set should be selected from the database and feed into the GA, as presented in Sec. §5.1.1. For this purpose, a set of 62 microelectrode recordings, one-third of signals from. 31.

(47) 1 Scaling Function Wavelet Function. 0.8. Wavele Function. 0.6 0.4 0.2 0 −0.2 −0.4 −0.6 −0.8 −1. 0. 10. 20. 30. 40. 50. Interpolated Samples. 60. 70. 80. Figure 6.3: Time response of the associated wavelet and scaling function (dot line) of the customized operators (u and p) after the off-line optimization procedure. Operator values are: u = [0.0278 − 0.2070 0.4292 0.4292 − 0.2070 0.0278], p = [0.0059 − 0.0488 0.2930 0.2930 − 0.0488 − 0.0059]. each class, is randomly selected. Due to possible over specialization of the GA, these signals should not be filtered to get either spikes or LFP components. Once the selection is done, the GA configuration parameters can be set as expressed in Sec. §5.1.5. Results for Nn = 6, Np = 6 is presented in Fig. 6.3 Above procedure depends on the database’s size and the convergence of the GA method. Therefore, mentioned experiment is repeated ten times for the following three concrete cases: (i) including a set containing 10% of the registers, termed small-sized database, (ii) 30% of the registers, termed medium–sized database, and (iii) 50% of the signals, termed large-sized database. Comparison values, given in terms of the fitness VDB , are shown in Table 6.1. Also, the effect of the partition size on the estimated filter frequency response is studied in Fig. 6.4. Table 6.1: Optimized LS operators for different input database sizes Size Filter Coefficients VDB 10% u = [0.1194 − 0.1539 0.2844 0.2844 − 0.1539 0.1194] 0.5626 p = [0.1771 − 0.2024 0.5253 0.5253 − 0.2024 0.1771] 30% u = [0.0278 − 0.2070 0.4292 0.4292 − 0.2070 0.0278] 0.6225 p = [−0.0584 − 0.0756 0.6340 0.6340 − 0.0756 − 0.0584] 50% u = [−0.2742 0.1121 0.4121 0.4121 0.1121 − 0.2742] 0.5884 p = [−0.0055 − 0.2421 0.7476 0.7476 − 0.2421 − 0.0055]. 32.

(48) (a). (b). (c). Figure 6.4: Low pass filter response associated to the operator u for different input database sizes on the optimization procedure. The Y-axis has been normalized. Current experiment is performed ten times. Outcomes are shown in terms of mean values and standard deviations (vertical lines). a) small-sized database. b) medium-sized database. c) large-sized database. Achieved outcomes evidence that both, the fitness criterion and the estimated operator’s value, are modified by the number of signals contained in the training set. Observed variations in the operator values tend to be stronger among cases, although attained fitness indexes are likely to be steady. Additionally, concerning the filter response, changes are also observable, namely, the more input signals the wider the pass-band. Therefore, filters lose specificity when large sets are used. On the contrary, filters might get overtraining due to small training databases. To sum up, the filters estimated for the 30% partition, which have exhibited a reasonable VDB -to-frequency response relation, are chosen to be applied in the following experiments. For sake of comparison, together with the SDFB customized wavelet, classification performance is also attained for several standard wavelets that are taken from the literature regarding the application in hand (Gemmar et al. [2008]), including wavelet functions of the families Daubechies, Symlet, and Cohen–. 33.

(49) Figure 6.5: Classification performance of the customized wavelet (blue line) and several existing wavelets (red line) including: Daubechies from order 2 to 7 (Db2, Db3, Db4, Db5, Db6, Db7), Symlet from order 3 to 7 (sym3, sym4, sym5, sym6, sym7), and Cohen–Daubechies–Feauveau of orders: cdf1.1, cdf1.3, cdf1.5, cdf2.2, cdf3.3. Notice that for the sake of visual comparison the standard deviation and classification performace of the customized wavelet is repetead on top of each classical wavelet. Daubechies–Feauveau (CDF). Classification average rates, obtained from each wavelet, are compared. As shown in Fig. 6.5, performance of considered standard wavelets turn to be lower than the one achieved by the customized wavelet. Particularly, differences ranged from 3% that is reached for the CDF family (order 1.5-3) up to 10% when using the rest of the considered standard wavelets. Lowest outcomes may be explained since families Daubechies, and Symlet do not adequately represent the microelectrode dynamic. On the contrary, similar rates are obtained with the CDF wavelets due to the features shared with the customized wavelet, e.i., linear phase, filter normalization, compact support.. 6.2.2. Discrimination Potential of LFP, Spikes, and Raw Microelectrode Recordings. The aim of this experiment is to determine the discrimination potential of LFP, spikes, and raw recordings using morphological features extracted from the SDFB decomposition coefficients. As presented in Sc. §5.1.3 different metrics may be extracted from the SDFB, however, only the best combination of three features is. 34.

(50) (a). (b). (c). Figure 6.6: 3D spaces obtained for (a) LFP, (b) spikes, and (c) raw microelectrode recording using the SDFB. The best combination of 3 features is the peak’s count (1) (2) (3) of w2 , the curve length of w2 , and the RMS value of w2 . Each class in the database is represented by a color. to be selected to become an alternative viewable 3D feature space. The selection is made offline based on classification performance. As a result, the peak’s count (1) (2) (3) of w2 , the curve length of w2 , and the RMS value of w2 became the best combination ( Eq. 5.5, Eq. 5.6, and Eq. 5.5). Feature spaces shown in Fig. 6.6 display typical behaviors of the LFP and Spike components of the treated nuclei. In particular, it can be seen that the quiet zone ZI is reliable confined to one corner of the 3D space, having low values on every axis. Since the feature measurements, generally, are presumed to have discriminatory power among ZI/quiet zone and STN/SubNR/TAL, the activity from these regions is distant from each other and very separable. Nonetheless, the SNR/SubNR zones display the most challenging classes for the classifier since they. 35.

(51) Table 6.2: Sensitivity (E) and Specificity (F) of the LFP, Spike, and raw microelectrode recording 3D spaces generated with the SDFB TAL STN SNr ZI LFP 3D E 65,00±7,82 41,25±18,68 32,50±13,86 100,00±0,00 S 90,00±3,62 87,12±6,65 91,27±4,68 93,25±2,9 Spike 3D E 95,00±4,08 85,00±16,43 82,50±7,30 100,00±0,00 S 97,87±2,01 97,80±1,40 96,36±2,84 100,00±0,00 Raw 3D E 96,50±3,37 88,75±12,43 85,83±7,91 100,00±0,00 S 97,87±2,01 98,47±1,25 97,45±2,60 100,00±0,00 are located in the higher corner of the 3D space, while the TAL zone is located in the middle zone. The LFP based 3D space, displayed in Fig. 6.6(a), have exhibited low RMS values, and low curve lengths, due to the elimination of the high frequency components of the microelectrode recordings. Furthermore, classes that typically present high spike activity (STN and SubNR patterns) are far from being easily separable on mentioned space. On the other hand, the Spike-based feature space (Fig. 6.6(b)), has shown an extension of the gap zone between neural and non neural classes, i.e., ZI and TAL/STN/SubNR, owning to the augmentation of the curve length value. For the last considered raw recording based space (Fig. 6.6(c)), intra-class dispersion is likely to increase, as well as, the global separation between classes. Further classification performance is assessed for mentioned 3D spaces. As the core of the proposed methodology is not the classification stage, but the characterization, a simple linear Bayesian classifier is used during the last stage because of its simplicity of implementation. The k -fold cross-validation approach is used to manage the database. Therefore, the database is randomly divided into two subsets: the training and validation. The former set comprises 146 patterns, while the remaining 67 patterns are related to the latter set, i.e., 70/30 partition. Also, there is no overlapping between sets. Then, signals are fed into the SDFB methodology for estimating above 3D feature spaces. Classification performance is given in terms of sensitivity (E) and specificity (S), corresponding to the rate of observations correctly classified and the rate of observations missed, respectively. As shown in Table 6.2, the most discriminant 3D space is the generated with the raw microelectrode signal. Besides, the number of missed observations in. 36.

(52) mentioned space is less than 3%, while the rate of correctly classified observations is 100%. Such findings suggest that splitting the microelectrode signals into its components of high and low frequency reduce the power of discrimination of the addressed nuclei in the SDFB based 3D spaces. Therefore, microelectrode recordings do not include preprocessing filtering during the following experiments.. 6.2.3. Classification Performance. In this section, results accomplished after using SDFB methodology are extended with ROC and statistical analysis, as shown in Table 6.3. The second column shows the Area-Under-the Curve (AUC) of the ROC when estimated for different couples of nuclei, which are typically found during the DBS trajectory to the STN (Wong et al. [2009]). An AUC of 1 represents a perfect test while an area of 0.5 represents a worthless test. Classes TAL/STN/SubNR and ZI present near perfect classification, besides, neural and non neural activity is well discriminated. Nevertheless, an expected result emerges in the TAL/ZI case. Fig. 6.6(c) depicts that both zones share the same position at the peak’s count axis since both nuclei present majority background neuronal noise. As result, the peak’s count feature failed the statistical test (p-value = 0.1981 two–tailed t-test), which is presented in the third column. For the third couple, corresponding to the STN and SubNR, test outcomes for each feature should be assumed as not statistical significant because of reduced representative data, that is, the low quantity of samples in both cases and the similar dynamics in both nuclei. However, as in the previous case, the 3D space allows the classifier to discriminate both classes with accuracy superior to 85%. Finally, performance under noise conditions is also measured. For sake of comparison, the SDFB with the customized wavelet and the best existing wavelet function is shown. Fig. 6.7 depicts the behavior of the proposed methodology under signal-to-noise ratios (SNR) from 1 dB to 30 dB. The performance remains steady regarding the wavelet used and the SNR for microelectrode recordings.. 37.

(53) Table 6.3: ROC and statistical analysis after approach Classes Feature AUC TAL vs Zi Peaks Count 0.539 Curve Length 1.000 RMS 1.000 STN vs Zi Peaks Count 0.932 Curve Length 1.000 RMS 1.000 STN vs SubNR Peaks Count 0.627 Curve Length 0.661 RMS 0.580 SubNR vs. Zi Peaks Count 0.953 Curve Length 1.000 RMS 1.000. employing the proposed SDFB p-value 0.198 0.000 0.000 0.000 0.000 0.000 0.035 0.009 0.136 0.000 0.000 0.000. 95% 0.447 1.000 1.000 0.887 1.000 1.000 0.488 0.526 0.437 0.905 1.000 1.000. C.I. 0.631 1.000 1.000 0.977 1.000 1.000 0.765 0.795 0.722 1.000 1.000 1.000. 1. Aevrage Classification Rate. 0.95. 0.9. 0.85. 0.8 Cohen−Daubechies−Feauveau 2.2 SDFB with Customized Wavelet 0.75 0. 1. 5. 10 15 20 Signal−to−Noise Ratio (dB). 25. 30. Figure 6.7: Classification performance under different noise-to-signal ratios. Signals are added with white Gaussian noise.. 6.3. Identification of Hand Movements by Means of EMG. Similar to previous experiment, 30% of the signals are randomly selected. Then, every pattern is decomposed up to level four to obtain approximation coefficients (1) (2) (3) (4) w2 , w2 , w2 , and w2 , respectively. Later, following the flow chart described in Sec. §5.2, a feature extraction procedure is carried out, namely, the curve length. 38.