Feature selection for spontaneous speech analysis to aid in Alzheimer’s disease diagnosis: A fractal dimension approach

Availableonlineatwww.sciencedirect.com

ScienceDirect

ComputerSpeechandLanguage30(2015)43–60

Feature selection for spontaneous speech analysis to aid in Alzheimer’s disease diagnosis: A fractal dimension approach

Karmele López-de-Ipi˜na

, Jordi Solé-Casals

, Harkaitz Eguiraun

, J.B. Alonso

, C.M. Travieso

, Aitzol Ezeiza

, Nora Barroso

,

Miriam Ecay-Torres

, Pablo Martinez-Lage

, Blanca Beitia

aDepartmentofSystemsEngineeringandAutomation,UniversityoftheBasqueCountry,Donostia,Spain

bDataandSignalProcessingResearchGroup,UniversityofVic–CentralUniversityofCatalonia,Vic,Spain

cResearchCenterforExperimentalMarineBiologyandBiotechnology,PlentziaMarineStation, UniversityoftheBasqueCountry,UPV/EHU,Plentzia,Bizkaia,Spain

dSignalandCommunicationsDepartment,TheInstituteforTechnologicalDevelopmentandInnovationonCommunications, UniversityofLasPalmasdeGranCanaria,LasPalmasdeGranCanaria,Spain

eNeurologyDepartmentCITA-AlzheimerFoundation,Donostia,Spain

fDepartmentofMathematics,UniversityoftheBasqueCountry,Vitoria-Gasteiz,Spain Received15October2013;receivedinrevisedform1August2014;accepted18August2014

Availableonline27August2014

Abstract

Alzheimer’sdisease(AD)isthemostprevalentformofdegenerativedementia;ithasahighsocio-economicimpactinWestern countries.ThepurposeofourprojectistocontributetoearlierdiagnosisofADandallowbetterestimatesofitsseveritybyusing automaticanalysisperformedthroughnewbiomarkersextractedthroughnon-invasiveintelligentmethods.Themethodselected isbasedonspeechbiomarkersderivedfromtheanalysisofspontaneousspeech(SS).Thusthemaingoalofthepresentworkis featuresearchinSS,aimingatpre-clinicalevaluationwhoseresultscanbeusedtoselectappropriatetestsforADdiagnosis.The featuresetemployedinourearlierworkofferedsomehopefulconclusionsbutfailedtocapturethenonlineardynamicsofspeech thatarepresentinthespeechwaveforms.Theextrainformationprovidedbythenonlinearfeaturescouldbeespeciallyusefulwhen trainingdataislimited.Inthiswork,thefractaldimension(FD)oftheobservedtimeseriesiscombinedwithlinearparametersin thefeaturevectorinordertoenhancetheperformanceoftheoriginalsystemwhilecontrollingthecomputationalcost.

©2014ElsevierLtd.Allrightsreserved.

Keywords: Nonlinearspeechprocessing;Alzheimer’sdiseasediagnosis;Spontaneousspeech;Fractaldimensions

∗Correspondingauthorat:DepartmentofSystemsEngineeringandAutomation,UniversityoftheBasqueCountry,PolytechnicSchool,Europa Plaza1,20008Donostia,Spain.Tel.:+34653723757.

E-mailaddresses:karmele.ipina@ehu.es(K.López-de-Ipi˜na),jordi.sole@uvic.cat(J.Solé-Casals),harkaitz.eguiraun@ehu.es(H.Eguiraun), jesus.alonso@ulpgc.es(J.B.Alonso),carlos.travieso@ulpgc.es(C.M.Travieso),aitzol.ezeiza@ehu.es(A.Ezeiza),nora.barroso@ehu.es (N.Barroso),mecay@cita-alzheimer.org(M.Ecay-Torres),pmlage@cita-alzheimer.org(P.Martinez-Lage),mariablanca.beitia@ehu.es(B.Beitia).

http://dx.doi.org/10.1016/j.csl.2014.08.002 0885-2308/©2014ElsevierLtd.Allrightsreserved.

1. Introduction

Alzheimer’sdisease(AD)isthemostcommontypeofdementiaamongtheelderly.Itischaracterizedbyprogressive andirreversiblecognitivedeteriorationwithmemorylossandimpairmentsinjudgmentandlanguage,togetherwith othercognitivedeficitsandbehavioralsymptoms.Thecognitivedeficitsandbehavioralsymptomsaresevereenoughto limittheabilityofanindividualtoperformeverydayprofessional,socialorfamilyactivities.Asthediseaseprogresses, patientsdevelopseveredisabilityandfulldependence.AnearlyandaccuratediagnosisofADhelpspatientsandtheir familiestoplanforthefutureandoffersthebestpossibilitiesoftreatingthesymptomsofthedisease.Accordingto currentcriteria,thediagnosisisexpressedwithdifferentdegreesofcertaintyaspossibleorprobableADwhendementia ispresentandotherpossiblecauseshavebeenruledout.ThediagnosisofdefiniteADrequiresthedemonstrationofthe typicalADpathologicalchangesatautopsy(McKhannetal.,1984,2011;VandePoleetal.,2005).Theclinicalhallmark andearliestmanifestationofADisepisodicmemoryimpairment.Atthetimeofclinicalpresentation,othercognitive deficitsarepresentinareaslikelanguage,executivefunctions,orientation,perceptualabilitiesandconstructionalskills (Morris,1993;APA,2000).Allthesesymptomsleadtoimpairedperformanceineverydayactivities.Approachesto theearlydiagnosisofADhaveinthepastfewyearsmadesignificantadvancesinthedevelopmentofreliableclinical biomarkers(AA,2014).

Despitetheusefulnessofbiomarkers,thecostandtechnologyrequirementsinvolvedmakeitimpossibletoapplysuch teststoallpatientswithmemorycomplaints.Giventheseproblems,non-invasiveintelligenttechniquesofdiagnosis maybecomevaluabletoolsforearlydetectionofdementia.Non-technicalstaffinthehabitualenvironmentsofthe patientcouldusethesemethodologies,whichincludee.g.automaticspontaneousspeechanalysis(ASSA)(Fig.1), withoutalteringor blocking the patients’ abilities, as the spontaneous speechinvolved inthese techniquesis not perceivedasastressfultestbythepatient.Moreover,thesetechniquesareverylow-costanddonotrequireextensive infrastructureortheavailabilityofmedicalequipment.Theyarethuscapableofyieldinginformationeasily,quickly, andinexpensively(Faundez-Zanuyetal.,2012;López-de-Ipi˜naetal.,2013a,b).

Inadditiontothe lossof memory,oneof themajor problemscausedbyADisthe lossof languageskills.We candetectdifferent communicationdeficits inthe area of language,including aphasia (difficulty inspeakingand understanding)andanomia(difficultyinrecognizingandnamingthings).Thespecificcommunicationproblemsthe patientencountersdependonthestageofthedisease(McKhannetal.,2011;VandePoleetal.,2005;Morris,1993):

1. Firststageorearlystage(ES):difficultyinfindingtherightwordinspontaneousspeech.Oftenremainsundetected.

2. Secondstageorintermediatestage(IS):impoverishmentoflanguageandvocabularyineverydayuse.

3. Thirdstageoradvancedstage(AS):answerssometimesverylimitedandrestrictedtoveryfewwords.

Fig.1.Signalandspectrogramofacontrolsubject(left)andasubjectwithAD(right)duringspontaneousspeech.

Themaingoalofthepresentworkisfeaturesearchinspontaneousspeechaimingatpre-clinicalevaluationforthe definitionoftestforADdiagnosis.Thesefeatureswillbeusedtodefinethegroup(CR)andthethreeADlevels.As phonologicalandarticulatoryimpairmentsmayoccuratpresentationor earlyinthecourseofAlzheimer’sdisease (Crootaetal.,2000),wewillusemeasuresthatcanreflectthenonlinearnatureofsuchchangesintheacousticalsignals.

Oneofthemostrelevantnonlineartechniquesforautomaticspeechrecognition(ASR)istheconsiderationofthefractal dimension(FD)ofthespeechsignalasafeaturetobeusedinthetrainingprocess.Ingeneral,fractaldimensionscanbe utilizedtoquantifythecomplexity,concerningthegeometryofadynamicalsystemgivenitsmultidimensionalphase- space.Thisquantificationisrelatedtotheactivedegreesoffreedomoftheassumeddynamicalsystem,providinga quantitativecharacterizationofasystem’sstate(PitsikalisandMaragos,2009;MaragosandPotamianos,1999).

Biologicalsystemsareregulatedbyinteractingmechanismsthatoperateacrossmultiplespatialandtemporalscales.

Theoutputvariablesofthesesystemsoftenhavecomplexfluctuationsthatarenotsolelyduetonoisebutalsocontain informationabouttheintrinsicdynamics.Timeseriesgeneratedbybiologicalsystemsmostlikelycontaindeterministic andstochasticcomponents(Costaetal.,2005).Classicalmethodsofsignalandnoiseanalysiscanquantifythedegree ofregularityofatimeseriesbyevaluatingtheappearanceofrepetitivepatterns,butmostsuchmethodsonlymodel linearcomponentswithoutintroducinganyinformationaboutnon-linearity,irregularitiesorstochasticcomponents.

Thiscomplex informationcould be essential when subtlechanges are analyzed.Thus,when appropriate dataare available,linearsystemscanbeimplementedfairlyrapidly,astheyrelyonwell-knownmachinelearningtechniques toachievetheirgoals,avoidingcomplexadjustmentstothesystem.

The interest in fractals in speechdate back to the mid-1980s (Pickover andKhorasani, 1986), andthey have beenusedforavarietyofapplications,includingconsonant/vowelcharacterization(Martinezetal.,2003;Langiand Kinsner,1995),speakeridentification(Nelwamondoetal.,2006),andend-pointdetection(Lietal.,2007),evenfor whisperedspeech(ChenandZhao,2006).Recentresearchconcernstheanalysisofpathologicalvoicesthroughafractal approach(Chouardetal.,2001;Ouayounetal.,1999;Péanetal.,2000,2002).Thefractaldimension(FD)quantifies theroughnessofatemporalsignalandestimatesitsdegreeoffreedom,andisthusagoodapproachtomodelingits fluctuations.Moreover,thefractalapproachallowsonetoquantifytheroughnessofthevoice,between1(sinusoidal complexsignal)and2(whitenoise).MaragosandPotamianos(1999)providesomemotivationandjustificationfrom thefieldofspeechaerodynamicsforusingfractaldimensiontoquantifythedegreeofturbulenceinspeechsignals.They alsoexploredifferencesinarticulatortrajectoriesandglobalintensity.InthisresearchFDoutperformszero-crossings analysisinthecaseofunvoicedsegments.LikeinotherworkonADdictionbasedonbiosignals(EEGandMEG) (Abásoloetal.,2008;Gómezetal.,2009)),thismethodologycouldbeusefulfordetectingsymptomsofAlzheimer’s, suchasdoubts,breaksorsilencesinthespeechsignal.Figs.2and3showdifferencesforacontrolsubject(CR)anda personwithADinintensity,pitchandFD.

Theapproachadoptedinthisworkistoimprovethesystemdevelopedinourpreviouswork(López-de-Ipi˜naetal., 2013a,b),augmentingthefeatureswithFD.TheFDisawell-knownandusedfeaturefordescribingthecomplexity ofasystem,andcouldhelpinthedetectionofsubtlechangesusefulforearlydiagnosis.Moreover,thisfeaturehasthe abilitytocapturethedynamicsofthesystemandthusrevealrelevantvariationsinspeechutterances.Moreprecisely, weuseimplementationsofHiguchi’s,Katz’sandCastiglioni’salgorithms(Higuchi,1988;Katz,1988;Castiglioni, 2010)inordertoaddthisnewfeaturetothesetthatfeedsthetrainingprocessofthemodel.

Theremainderofthispaperisorganizedasfollows:inSection2,thematerialsarepresented.Section3explains themethodologyoftheexperiments,Section4showstheexperimentalresults,andfinally,conclusionsarepresented inSection5.

2. Materials

ThisstudyisfocusedonearlyADdetectionanditsobjectiveistheidentificationofADinthepre-clinical(before firstsymptoms)andprodromic(someveryearlysymptomsbutnodementia)stages.Theresearchpresentedhereisa complementarypreliminaryexperimenttodefinethresholdsforanumberofbiomarkersrelatedtospontaneousspeech.

Thefeaturesearchinthisworkisdesignedforpre-clinicalevaluationtoselectappropriatetestsforADdiagnosis.The dataobtainedwillcomplementthebiomarkersetofeachpersonindiagnosingAD.

In anattempttodevelopanewmethodologyapplicable toawiderangeof individualsdifferingwithregardto sex,age,languageandculturalandsocialbackground,wehaveconstructedamulticulturalandmultilingual(English, French,Spanish,Catalan,Basque,Chinese,ArabicandPortuguese)databasewithvideorecordingsof50healthyand

Fig.2.Analysisofacontrolperson(CR).UtteranceinBasquelanguage,“...Belerobat...”:(a)speechsignal,pitch(blue)andintensity(yellow), (b)spectrogram,(c)Higuchifractaldimension(CFD)and(d)Castiglionifractaldimension(HFD).(Forinterpretationofthereferencestocolorin thetext,thereaderisreferredtothewebversionofthisarticle.)

20ADpatients(i.e.,patientswithapriordiagnosisofAlzheimer’s)recordedfor12hand8h,respectively.Theage spanoftheindividualsinthedatabasewas20–98yearsandtherewere20malesand20females.Thisdatabaseiscalled AZTIAHO.Alltheworkwasperformedinstrictaccordancewiththeethicalguidelinesoftheorganizationsinvolved intheproject.

Therecordingsconsistedofvideosofspontaneousspeech–peopletellingpleasantstoriesorrecountingpleasant feelingsaswellasinteractingwitheachotherinfriendlyconversation.Therecordingatmospherewasrelaxedand non-invasive.TheshorterrecordingtimesfortheADgroupareduetothefactthatADpatientsfindspeechmoreof

Fig.3.AnalysisofapersonwithAD(AD),utteranceinSpanish,“Noseloqueeseso...”:(a)speechsignal,pitch(blue)andintensity(yellow),(b) spectrogram,(c)Higuchifractaldimension(CFD)and(d)Castiglionifractaldimension(HFD).(Forinterpretationofthereferencestocolorinthe text,thereaderisreferredtothewebversionofthisarticle.)

aneffortthanhealthyindividuals:theyspeakmoreslowly,withlongerpauses,andwithmoretimespentonlooking forthecorrectwordandutteringspeechdisfluenciesorbreakmessages.Intheadvancedstageofthedisease,theyfind thisefforttiringandoftenwanttostoptherecording.Wecompliedwiththeirrequests.Thevideowasprocessedand theaudioextractedinwavformat(16bitsand16kHz).Thefirststepwasremovingnon-analysableevents:laughter, coughing, shorthardnoisesandsegmentswherespeakersoverlapped.Next,backgroundnoisewasremovedusing denoiseradaptivefiltering.Afterthepre-processing,about80%ofthematerialfromthecontrolgroupand50%ofthe materialfromtheADgroupremainedsuitableforfurtheranalysis.Thecompletespeechdatabaseconsistsofabout 60minof materialfor the ADgroup andabout9h for thecontrol. Thespeechwasnextdivided into consecutive

segmentsof60sinordertoobtainappropriatesegmentsforallspeakers,resultingfinallyinadatabaseofabout600 segmentsofspontaneousspeech.

Finally,inordertoperformourexperimentsasubsetof20ADpatientswasselected(68–96yearsofage,12women, 8men)withadistributioninthethreestagesofADasfollows:firststage[ES=4],secondarystage[IS=10]andtertiary stage[AS=6].Thecontrolgroup(CR)wasmadeupof20individuals(10maleand10female,aged20–98years)and representingawiderangeofspeechresponses.ThissubsetofthedatabaseiscalledAZTIAHORE.

3. Methods

3.1. Featureextraction

3.1.1. Featuresselectedforautomaticspontaneousspeechanalysis(ASSA)

Spokenlanguageisoneofthemostimportantelementsdefininganindividual’sintellect,sociallife,andpersonality;

itallowsustocommunicatewitheachother,shareknowledge,andexpressourculturalandpersonalidentity.Spoken languageisthemostspontaneous,natural,intuitive,andefficientmethodofcommunicationamongpeople.Therefore, theanalysisbyautomatedmethodsofspontaneousspeech(SS–freeandnaturalspokencommunication),possibly combinedwithothermethodologies,couldbeausefulnon-invasivemethodforearlyADdiagnosis(López-de-Ipi˜na etal.,2013a,c).Theanalysisofspontaneousspeechfluencyisbasedonthreefamiliesoffeatures(SSFset),obtained bythePraatsoftwarepackage(Praat)andsoftwarethatweourselvesdevelopedinMATLAB.Forthatpurpose,an automaticVoice ActivityDetector(VAD)(VAD;SoléandZaiats,2003)hasextractedvoiced/unvoicedsegments as partsofanacousticsignal(López-de-Ipi˜naetal.,2013a,b).

Thethreefamiliesoffeaturesinclude(López-de-Ipi˜naetal.,2013b):

(1) Duration:thehistogramcalculatedoverthemostrelevantvoicedandunvoicedsegments,theaverageofthemost relevantvoiced/unvoiced,voiced/unvoicedpercentageandspontaneousspeechevolutionalongthetimedimension, andthevoicedandunvoicedsegments’mean,maxandmin;

(2) Timedomain:shorttimeenergy;

(3) Frequencydomain,quality:spectralcentroid.

Theenergyofasignalistypicallycalculatedonashort-timebasis,bywindowingthesignalataparticulartime, squaringthesamplesandtakingtheaverage.Thespectralcentroidiscommonlyassociatedwiththemeasureofthe brightnessofasound.Thismeasureisobtainedbyevaluatingthe“centerofgravity”usingtheFouriertransform’s frequencyandmagnitudeinformation.

3.1.2. Fractaldimension

Mostfractalsystemshaveacharacteristiccalledself-similarity.Anobjectisself-similarifaclose-upexamination of the objectreveals that it is composedof smaller versionsof itself.Self-similarity can bequantified as a rela- tivemeasureof thenumberofbasicbuildingblocksthatformapattern,andthismeasureisdefinedas thefractal dimension.

ThereisnotanyprecisereferenceoftheFDvalueagivenwaveformshouldhave.Inaddition,speechwaveforms arenotstationary,somostASRtechniquesemployshortsectionsofthesignalinordertoextractfeaturesfromthe waveform.Thismeansthatoneplausibletechniqueforextractingfeaturesfromspeechwaveforms,forthepurpose ofrecognizingdifferentphonemes,istodividethesignalinshortchunksandcalculatethefeaturesforeachchunk.

Thiswasthe approach weadopted. In otherwords, we calculatedthe fractal dimensionof short segments of the waveformandobservedtheevolutionoftheobtainedvaluesalongthewholesignal,withtheaimoffindinginitfractal characteristicsthatcouldhelpinidentifyingdifferentelementsofthespokenmessage.

Thereareseveralalgorithmsformeasuringthefractaldimension.Inthecurrentworkwefocusonthealternatives thatareespeciallysuitedfortimeseriesanalysisanddonotrequirepreviousmodelingofthesystem.Threeofthese algorithmsareHiguchi(1988),Katz(1988)andCastiglioni(2010),namedfortheirauthors.HiguchiandCastiglioni werechosenbecausetheyhavebeenreportedtobemoreaccurateinpreviousworkwithunder-resourcedconditions (Ezeizaetal.,2013;López-de-Ipi˜naetal.,2013b,c).Katzisalsoreportedasarobustalgorithmtocalculatefractal dimension(Estelleretal.,2001).

Higuchi(1988)proposedanalgorithmformeasuringthefractaldimensionofdiscretetimesequencesdirectlyfrom thetimeseriesx(1),x(2),...,x(N)Thealgorithmisbasedonanewtimeseries,x^k_m,constructedfromtheoriginalone, asfollowing:

x^k_m=



x(m),x(m+k),x(m+2k),...,x

 m+

N−m k

 k



, for m=1,2,...,k (1) wherea meansintegerpartfora,mindicatestheinitialtimevalueandkindicatesthediscretetimeintervalbetween points(delay).Foreachofthetimeseriesx^k_mconstructed,theaveragelengthLm(k)iscomputedas:

L_m(k)=

 _N−m

k

i=1 |x(m+ik)−x(m+(i−1)k)|(N−1) _N−m

k

k (2)

Next,thelengthofthecurveforthetimeintervalkisdefinedasthesumvalueoverksetsofLm(k)asshowedin Eq.(3):

L(k)=^k

m=1

L_m(k) (3)

Finally,theslopeofthecurveln(L(k))/ln(1/k)isestimatedusingleastsquareslinearbestfit,andtheresultisthe Higuchifractaldimension(HFD).

Ontheotherhand,Katz(1988)proposedanormalizedformulaofthefractaldimension(seeEq.(3))becausethis fractaldimensiondependsontheparticularunitsofmeasure.

TheFDofaplanarcurvecanbedefinedas:

FD= log₁₀(L)

log₁₀(d) (4)

InthiscaseListhelengthofthecurveanddisthediameter(theplanarextent)ofthecurve.Fortimeseries,which areorderedsetsof(x,y)pointpairs,thetotallengthLisnothingbutthesumofthedistancesbetweensuccessivepoints, aspresentedinEq.(5):

L=ⁿ

i=1

l_i,i+1 (5)

whereli,jmeanstheEuclideandistancebetweentwopointpairs(xi,yi)and(xj,yj)asshowninEq.(5),andn+1isthe totalnumberofpointpairs.

l_i,j=

(yi−y_j)²+(xi−x_j)² (6)

Thediameterofthetimeseriescanbecalculatedasthefarthestdistancebetweenthestartingpoint(point1)and anyotherpoint(pointi)ofthetimeseries,aspresentedinEq.(6):

d =max{l1,i} (7)

TheFDcomparestheactualnumberofunitsinacurvewiththeminimumnumberofunitsrequiredtoreproduce apatternofthesamespatialextent;andasmentionedabove,willdependuponthemeasurementunitsusedbecause FDiscalculatedinadiscretespacenotincontinua.Katz’sapproachsolvesthisproblembynormalizingdandLby thelengthoftheaveragestepa,theaveragedistancebetweensuccessivepoints,definedasL/n,wheren+1isthetotal numberofpointpairs(aspresentedinEq.(5)).Therefore,normalizingdistancesin(4)weobtainKatz’sapproachto calculatetheFDofatimeseries(8):

FD= log₁₀(n)

log₁₀(n)+log₁₀(d/L) (8)

However,Castiglioni(2010)claimsthatintheX–Yplane,whichdescribesthewaveform,themagnitudesxandyin useareintrinsicallydifferentsincetheycorrespondtothemagnitudeofthesignal(y)andtimepoints(x).Therefore, giventhat the inputsignal isamono-dimensional waveform,the length andthe extensioncanbe rewrittenusing

Fig.4.Higuchifractaldimension(HFD)foracontrolsubject(above)andanADsubject(below),fordifferentwindowsizes.

Mandelbrot’sapproach.Asimpleandefficientwaytodothisistomeasurethesetwomagnitudesdirectlyintheirown dimension.Foreach,theextensiononthey-axisistherangeofyk:

d=max{y_k}−min{y_k} (9)

andthelengthListhesumofalltheincrements,inmodulus:

L=ⁿ

k=1

|y_k+1−y_k| (10)

Whereagain,disthediameterofthecurve,Listhelengthofthecurveandn+1isthetotalnumberofpointpairs.

3.1.3. Windowsizeduringthefeatureextractionprocess

Theselectionofanappropriatewindowsizetobeusedduringtheexperimentsisessential.Fig.4illustratesthe qualitativeeffectofwindowsizeontheresultsoftheexperiments.Broadlyspeaking,thefractaldimensionisatool forattemptingtocapturethedynamicsofthesystem.Withashortwindow,theestimationishighlylocalandadapts fasttothechangesinthewaveform.Whenthewindowislonger,somedetailsarelostbutthefractaldimensionbetter anticipatesthecharacteristicsofthesignal.Additionally,previousstudiesthattakeintoaccountthewindowsizeof similardimensionestimations(Tsonis,2007;Jang,2011;Estelleretal.,2001)suggestthatalongerwindowcouldbe usefulinsomecases.Consequently,fourwindow-sizesof160,320and1280pointswillbeanalyzed.

Table1

Accuracy(%)inthefinalsystemwithMLPandKNNforCRandAD.

Nameoftheset Description

SSF Spontaneousspeechfeatures

SSSF Automaticselectionofspontaneousspeechfeatures SSSF+FD SSSFsetandfractaldimension(FD)

VFD Maximum,minimum,variance,standarddeviation,medianandmodeaverageforfullsignalandvoicedsignal SVFD Automaticselectionofmaximum,minimum,variance,standarddeviation,medianandmodeaverageforfullsignaland

voicedsignal

Table2

Accuracy(%)inthefinalsystemwithMLP(SSSF+CSVFD)andKNN(SSSF+HSVFD)forCRandAD(WS=320)andconfidenceintervalfor 95%and90%.

Class KNN MLP

Acc CI(95%) CI(90%) Acc CI(95%) CI(90%)

CR 90.90 [84.09,97.90] [85.21,96.79] 92.43 [86.85,99.15] [87.83,9816]

AD 87.30 [79.07,95.52] [80.39,94.20] 90.9 [84.10,98.03] [85.06,96.93]

3.2. Automaticfeatureselection

Inordertodevelopapplicationswithlowcomputationalcost,featuresetsshouldbeoptimized.Inafirstapproach, threealgorithmswillbeusedforautomaticattributeselection.Specifically,thefollowingonesfromtheWEKAsoftware (WEKA)suitewereselectedtotakeintoaccountgainrelativetoeachclass,eachfeatureandtherelationsamongthem:

1. SVMAttributeEval:EvaluatestheusefulnessofanattributebyusinganSVMclassifier.Attributesarerankedby thesquareoftheweightassignedbytheSVM.Attributeselectionformulticlassproblemsishandledbyranking attributesforeachclassseparatelyusingaone-versus-allmethodandthendealingfromthetopofeachpiletogive afinalranking(WEKA).

2. CfsSubsetEval:Evaluatestheusefulnessofasubsetofattributesbyconsideringtheindividualpredictiveabilityof eachfeature,alongwiththedegreeofredundancybetweenthem.Subsetsoffeaturesthatarehighlycorrelatedwith theclasswhilefeatureshavinglowintercorrelationarepreferred(WEKA).

3. GainRatioAttributeEval:Evaluatestheusefulnessofanattributebymeasuringthegainratio withrespecttothe class(WEKA).

Thesealgorithmswillbeusedovertheinitialfeaturesetstoselectsmallerandappropriateattributesetsfor low computationalcostperformance.

3.3. Featuresets

Intheexperimentation,threefamiliesoffeaturesetswillbeused:

1. SSFandSSSF 2. SSSF+FD

3. SSSF+VFDandSSSF+SVFD

Fromthesethreefamilieswederivefivedifferentsetsthatwillbeusedinthefollowingexperiments,asdetailedin Table1(seealsoTable3foracompletelistofacronymsusedinthiswork).

3.4. Automaticclassification

Themaingoalofthepresentworkisfeaturesearchinspontaneousspeechorientedtopre-clinicalevaluationfor thedefinitionoftestsforADdiagnosis.ThesefeatureswilldefinetheCRgroupandthethreeADlevels.Asecondary goalwillbethecontrolofcomputationalcostorientedtoreal-timeapplications.Thusautomaticclassificationwillbe modeledwithaviewtowardefficiency.Twodifferentparadigmswillbeevaluated:

1. Multilayerperceptron(MLP)withneuronnumberinhiddenlayer(NNHL)=(attribute/number+classes/number) andtrainingstep(TS)NNHL×10,MLPA500.

2. k-Nearestneighbors(KNN)paradigm.

TheWEKAsoftwaresuitehasbeenusedincarryingouttheexperiments.TheresultswereevaluatedusingClas- sificationErrorRate(CER),Accuracy(Acc)andCumulativeError(Arias-Londo˜noetal.,2010;Godinoetal.,2005, 2006).Forthetrainingandvalidationsteps,weusedk-foldcross-validationwithk=10.Cross-validationisarobust validationmethod forvariable selection(Picard andCook,1984).Repeatedcross-validation (ascalculatedby the WEKAenvironment)allowsrobuststatisticaltests.WealsousethemeasurementprovidedautomaticallybyWEKA

“Coverageofcases”(0.95level)andconfidenceintervalforpercentages(CI)for95%,90%and80%.

4. Experimentalresults

TheexperimentationwascarriedoutwithAZTIAHOREdataset.

4.1. Analysisofthefractaldimension

Inthisfirststageabout80attributesweregeneratedwithregardtothecriteriadescribedabove:featuresselectedfor automaticspontaneousspeechanalysis(ASSA)andfeaturesbasedonthefractaldimension.Then,automaticfeature selectionwascarriedoutusingthemethodologydescribedin3.2forallthreefractaldimensionalgorithmsandfour differentwindow-sizes(160,320,640and1280points).

4.2. Automaticfeatureselection

AutomaticfeatureselectionwascarriedoutusingthemethodologydescribedinSection3.2.Forallthreefractal dimensionalgorithmsandfourdifferentwindow-sizes(160,320,640and1280points),12similarproposalsoffeature setswereanalyzed.About50featureswereselectedfromtheoriginalsets.Theselectedattributesarethesameforall cases:featuresrelativetoASSAandfeaturesbasedonthefractaldimension.Finallyfollowingfeaturefamilieswere usedinthenextSection4.3:SSF,selectionofSSF(SSSF),SSSF+FD,fullsetsoffeatures(SSSF+VFD)andsets withtheselectedfeatures(SSSF+SVFD).SSSFsetoutperformstheCERratesin%ofSSF(MLP,24.81%andKNN, 29.46%)inabout5%.Specifically,thereferenceresultsCERin%forSSSF,withoutFD,are:MLP,20.16%andKNN, 27.14%(seeRefbluebarinFig.5).RecognitionratesforbothalgorithmsareoptimalfortheCRgroupbutverypoor forES.ThusSSSFwasselectedforuseinthesubsequentexperimentsdescribedbelow.

4.3. Automaticclassification

InthenextstagethemethodologydescribedinSection3.4wasused.Thetaskwasautomaticclassification,withthe classificationtargetsbeing:healthyspeakerswithoutneurologicalpathologiesandspeakersdiagnosedwithAD.Four kindsofexperimentswerecarriedout:selectionofwindow-size;featureanalysis;analysisofglobalresults;analysis ofclasses’results.

Fig.5showstheclassificationerrorratein%forfractaldimensionalgorithmsandbothparadigms:MLPandKNN.

Inthistask,severalwindowsizeswereanalyzed:160,320,640and1280points.TheSSSFsetandfractaldimension setswereusedinthisstage.TheminimumCERisachievedwithMLPandwindowsofmiddlesize:320and640.

Bothwindow-sizeswill beused insubsequent experiments.Most ofthe newproposals,whichincludethe fractal dimension,achievedimprovementsoverthereferencemethods.Fig.6analysestheinfluenceofattributeselectionwith

Fig.5.Classificationerrorrate(%)forfractaldimensionalgorithms,severalwindowsizesandbothparadigms:MLPandKNN.Refbarswere obtainedwithoutusingfractaldimensionfeatures.

Fig.6.Classificationerrorrate(%)forseveralwindowsizesandbothparadigms:MLPandKNNwithfractaldimensionfeaturesets(VFD)and attributesetsafterselection(SVFD).Refbarswereobtainedwithoutusingfractaldimensionfeatures.

Fig.7.Classificationerrorratein%forthethreedefinedfractaldimensionalgorithmswithonlyFDandwithsetsafterattributeselection(SVFD) andforbothparadigmsMLPandKNN.Refbarswereobtainedwithoutusingfractaldimensionfeatures.

regardtofractaldimensionalgorithms,modelingandwindowsize.WithMLPtheCERin%decreasedinallcases.

RegardingKNN,thebestglobalresultswereobtainedfortheHiguchialgorithm.Thecomputationalcostinallcases issignificantlyreducedwiththeKNNparadigm (seeFig.10).Thus,theSSSF+SVFDsets,withfeaturesselected byautomaticselection,wereusedinsubsequentexperiments.Theglobalresultsofthisstudyweresatisfactory.The newfractalfeaturesdecreaseCERinMLPfrom20.16%to14.73%andinKNNfrom27.14%to21.71%(seeFig.6).

Thissuggeststhatthesetsthatincludethefractaldimension,itsdetailedvariationsandautomaticattributeselection, wereabletoproperlymodelnon-linearsignalfeatures.Inbothexperimentsthecoverageofcaseswasabout94%.The confidenceintervalsareofabout±3forMLPand±5forKNNover80%.Fig.7showstheimprovement(CERin%)in mostcaseswhenSVFDisincluded,mainlyforMLP.ThebestresultsareachievedwithMLPforKatzandCastiglioni withawindow-sizeof320points.Ingeneral,KNNislessstablethanMLP.Moreover,MLPpresentsgoodratesfor computationalcost(see Fig.10).KNN hasthelowestcomputationalcostforallcasesandwithfractal dimension featuresisabletodetectESsegments,butcannotachievetheresultsobtained byMLP.Thebestglobalresultsare producedbySSSF+CSVFD(14.73%)withMLPandWS=320.KNNobtainsthebestresultswithHiguchialgorithm variants(21.71%).

Fig.8 showstheresultsforclasses (accuracyin%). SSSF+CSVFDwithMLPandWS=320isagain thebest option.Thissetobtainsthebestresultsforallclassesandalsoimprovestheclassificationwithregardtoearlydetection (ESclass).InthecaseofIS,wealsoobtainedabetterrateofidentifyingthemiddleADlevel.Themodelisalsoableto discriminatebetweenpathologicalandnon-pathologicalsegmentsineachpatient.Finally,regardingcumulativeerror forclassesin%(CE),Fig.9showstheobtainedresults.TheW320option,whichpresentsthelowestCE,seemsto bettercapturethedynamicsof thesignalwiththevariantsofCastiglionifractaldimension.Itshouldbenotedthat thereisasmallpercentageoffalsepositives,whichmayduetodoubtsinutteranceproduction.Coverageofcasesis about95%intheglobalsystem.ThedifferencesbetweenESandtheotherstageswithregardtoratesisduetothe factthatthespecificcommunicationproblemsthepatientencountersdependonthestageofthedisease,asdescribed inSection1(aphasia,anomia).Moreover,whenpatientsareintheESorASstage,problemsrelatedtoage–suchas dysphonicvoice–areoftenpresentaswell,asagingcausesalterationsinthevoiceproductionsystem.Table2shows accuracy(%)inthefinalsystemwithMLPandKNNforCRandADandalsoconfidenceintervalfor95%and90%.

Fig.8.Accuracy(%)withMLPandKNNfordifferentclasses:CR,ES,ISandAS.Refbarswereobtainedwithoutusingfractaldimensionfeatures.

Fig.9. Cumulativeerror(%)withMLPandKNNfordifferentclasses:CR,ES,ISandAS.

Table3 Acronymsused.

Acronym Description

AD Alzheimer’sdisease

CR Control

FD Fractaldimension

KFD KatzFD

HFD HiguchiFD

CFD CastiglioniFD

SSF Spontaneousspeechfeatures

SSSF SelectionofSSF

SSSF+FD SSSFsetandHFD,KFDorCFD

STD Standarddeviation

VFD Maximum,minimum,standarddeviation,medianandmodeaverageforfullandvoicedsignal

SVFD AutomaticselectionofVFD

MLP Multilayerperception

NNHL Neuronnumberinhiddenlayer

TS Trainingstep

MLP501000 NNHL=50andTS=1000

MLPA500 NNHL=max(attribute/number+classes/number)andTS=NNHL×10

KNN k-Nearestneighbors

CER Classificationerrorrate

Acc Accuracy

CE Cumulativeerror

Fig.10analysesthecomputationalcostoftheproposedalgorithm:(a)withregardtoModelBuildingandRecognition Processfor:KNN,MLPA500andMLP501000(MLPwithNNHL=500andTS=1000);and(b)withregardtoModel BuildingandRecognitionProcessfor:KNN,andMLPA500withdifferentfeaturesset.

Wecanobserve (Fig.10a)that whenbuildingthemodel,KNNandMLPA500havesimilarcomputationalcost, followedbyMLP501000;butinclassificationtasks,clearlyKNNisthebestoneregardingthisaspect.Whatismore interestingistoobserveindetailFig.10b,wherewepresentalsocomputationalcostforbuildingthemodelandfor classificationtask,butinthiscaseonlyforKNNandMLPA500,andwithdifferentsetsoffeatures.Now,evenifstill KNNoutperformsMLP,automaticselectionofparametersallowsustoclearlydiminishthecomputationaltimefor MLPA500(about4timeslessthanMLP501000),whichinturnsisthemethodthatgivesbetterperformanceinterms ofClassificationErrorRate(seeFigs.6–8).Therefore,MLPwithautomaticselectionofparametersisthebestoption becauseitobtainsthelowestCERandareasonablecomputationaltimefor therecognitionprocess(less than10s, whichissuitableforrealtimeapplications).

ComputationalcostwithregardtoModelBuildingandRecognitionProcessfor:KNNandMLPA500withdifferent featuresets

Fig.10.(a)Computationalcostwithregardtomodelbuildingandrecognitionprocessfor:KNN,MLPA500andMLP501000(MLPwithNNHL=500 andTS=1000),(b).

5. Conclusionsandfuturework

Themaingoalofthepresentprojectisfeaturesearchinspontaneousspeechtoaidinthepre-clinicalevaluation andtheselectionofappropriatetestsforADdiagnosis.Thesefeaturesareofgreatrelevanceforhealthspecialiststo identifyADsufferersanddifferentiatebetweenthethreeADlevels.Theapproachofthisworkistoimproveprevious modelingbasedonspontaneousspeechfeatureswithfractaldimensions.Withregardtopreviouswork,newmodeling applicationswithsmallcomputationalcosthavebeenevaluated.Moreprecisely,weproposetouseHiguchi’s,Katz’s, andCastiglioni’salgorithmsinordertoaddthesenewfeaturestothesetthatfeedsthetrainingprocessofthemodel.

Inthiswork,anapproachtotheinclusionofnonlinearfeaturesisdescribed.Thisstraightforwardapproachappears robustintermsofcapturingthedynamicsofthewholewaveform,astheCERdecreaseswhenusingFDfeatures,with lesscomputationalcostthanthepreviouslyusedones.Infutureworkwewillintroducenewfeaturesrelatedtospeech modelingthatcanpotentiallybeappliedtostandardmedicaltestsforADdiagnosisandtoemotionresponseanalysis.

Wewill alsomodel the fractaldimension using otheralgorithms,andentropyfeatures as newnonlinear features.

Finally,anewapproachbasedonone-classclassifierdesignedforearlydetectionwillbealsodeveloped.

Acknowledgments

ThisworkhasbeenpartiallysupportedbyaSAIOTEKfromtheBasqueGovernment,UniversityofVic–Central UniversityofCataloniaundertheresearchgrantR0947,andtheSpanishMinisteriodeCienciaeInnovaciónTEC2012- 38630-C04-03.

References

AA,2014.Alzheimer’sAssociation,Availablefrom:http://www.alz.org/

AmericanPsychiatricAssociation(APA),2000.DiagnosticandStatisticalManualofMentaldisorders,WashingtonDC,4thed.,TextRevision.

Abásolo,D.,Escudero,J.,Hornero,R.,Espino,P.,Gómez,C.,2008.FractaldimensionoftheEEGinAlzheimer’sdisease.Encycl.Healthcare Inform.Syst.

Arias-Londo˜no, J.D., Godino-Llorente, J.I., Sáenz-Lechón, N., Osma-Ruiz, V., Castellanos-Domínguez, G., 2010. An improved method for voice pathology detection by means of a HMM-based feature space transformation. Pattern Recogn. 43 (9), 3100–3112, http://dx.doi.org/10.1016/j.patcog.2010.03.019.

Castiglioni,P.,2010.WhatiswronginKatz’smethod?Commentson:“Anoteonfractaldimen-sionsofbiomedicalwaveforms”.Comput.Biol.

Med.40,11–12.

Chen,X.,Zhao,H.,2006.Fractalcharacteristic-basedendpointdetectionforwhisperedspeech.In:Proceedingsofthe6thWSEASInternational ConferenceonSignal,SpeechandImageProcessing,Lisbon,Portugal.

Chouard,C.H.,Pean,V.,Ouayoun,M.,Meyer,B.,2001.Afractalapproachtothefeaturesofspeechconsonants.ActaOtolaryngol.121(January (2)),249–253.

Croota,K.,Hodgesa,J.R.,Xuerebc,J.,Pattersona,K.,2000.PhonologicalandarticulatoryimpairmentinAlzheimer’sdisease:acaseseries.Brain Lang.75(2),277–309.

Esteller,R.,Vachtsevanos,G.,Echauz,J.,Litt,B.,2001.Acomparisonofwaveformfractaldimensionalgorithms.IEEETrans.CircuitsSyst.I:

Fundam.TheoryAppl.48(2),177–183.

Ezeiza,A.,LópezdeIpi˜na,K.,Hernández,C.,Barroso,N.,2013.Enhancingthefeatureextractionprocessforautomaticspeechrecognitionwith fractaldimensions.Cogn.Comput.5,545–550,http://dx.doi.org/10.1007/s12559-12-91-65.29120.

Faundez-Zanuy, M., et al., 2012. Biometric applications related to human beings: there is life beyond security. Cogn. Comput., http://dx.doi.org/10.1007/s12559-012-91699.

Godino,J.I.,Gómez-Vilda,P.,Blanco-Velasco,M.,2006.Dimensionalityreductionofapathologicalvoicequalityassessmentsystembasedon Gaussianmixturemodelsandshorttermcepstralparameters.IEEETrans.Biomed.Eng.53(10),1943–1953.

Godino,J.I.,Gómez-Vilda,P.,Sáenz-Lechón,N.,Blanco-Velasco,M.,Cruz-Roldán,F.,Ferrer-Ballester,M.A.,2005.SupportVectorMachines AppliedtotheDetectionofVoiceDisorders.SpringerVerlag,BerlinHeidelberg,pp.219–230.

Gómez,C.,Mediavilla,A.,Hornero,R.,Abásolo,D.,Fernández,A.,2009.UseoftheHiguchi’sfractaldimensionfortheanalysisofMEGrecordings fromAlzheimer’sdiseasepatients.Med.Eng.Phys.31(3),306–313,ISSN1350-4533.

Higuchi,T.,1988.Approachtoanirregulartimeseriesonthebasisofthefractaltheory.Phys.D31277,283.

Jang,J.S.R.,2011.AudioSignalProcessingandRecognition.In: RogerJang’sHomepage,Availablefrom:http://www.cs.nthu.edu.tw/∼jang (accessed2011).

Katz,M.,1988.Fractalsandtheanalysisofwaveforms.Comp.Biol.Med.18(3),145–156.

Langi,A.,Kinsner,W.,1995.Consonantcharacterizationusingcorrelationfractaldimensionforspeechrecognition.In:ProceedingsofCommuni- cations,Power,andComputing.ConferenceProceedings.IEEE.

Li,Y.,Fan,Y.,Tong,Q.,2007.Endpointdetectioninnoisyenvironmentusingcomplexitymeasure.In:Proceedingsofthe2007International ConferenceonWaveletAnalysisandPatternRecognition,Beijing,China.

López-de-Ipi˜na,K.,Alonso,J.B.,Solé-Casals,J.,Barroso,N.,Faundez-Zanuy,M.,Travieso,C.,Ecay-Torres,M.,Martinez-Lage,P.,Egiraun, H.,2013b.OnautomaticdiagnosisofAlzheimer’sDiseasebasedonSpontaneous,SpeechAnalysisandEmotionalTemperature,Cognitive Computation.Springer,Berlin,http://dx.doi.org/10.1007/s12559-013-9229-9.

López-de-Ipi˜na,K.,Eguiraun,H.,Solé-Casals,J.,Ecay,M.,Ezeiza,A.,Barroso,N.,Martinez-Lage,P.,Martinez-de-Lizardui,U.,2013c.Feature extractionapproachbasedonfractaldimensionforspontaneousspeechmodellingorientedtoAlzheimerdiseasediagnosis.In:NOLISP,2013, pp.144–151.

López-de-Ipi˜na,K.,Alonso,J.B.,Travieso,C.M.,Solé-Casals,J.,Egiraun,H.,Faundez-Zanuy,M.,Ezeiza,A.,Barroso,N.,Ecay,M.,Martinez- Lage,P.,Martinez-de-Lizardui,U.,2013a.Ontheselectionofnon-invasivemethodsbasedonspeechanalysisorientedtoautomaticAlzheimer diseasediagnosis.Sensors13(5),6730–6745.

Costa,M.,Goldberger,A.L.,Peng,C.K.,2005.Multiscaleentropyanalysisofbiologicalsignals.Phys.Rev.E71(2pt.1).

Maragos,P.,Potamianos,A.,1999.Fractaldimensionsofspeechsounds:Computationandapplicationtoautomaticspeechrecognition.J.Acoust.

Soc.Am.105(3),1925–1932.

Martinez,F.,Guillamon,A.,Martinez,J.J.,2003.Vowelandconsonantcharacterizationusingfractaldimensioninnaturalspeech.In:Proceedings ofNOLISP’2003.

McKhann,G.,Drachman,D.,Folstein,M.,Katzman,R.,Price,D.,Stadlan,E.M.,1984.ClinicaldiagnosisofAlzheimer’sdisease:Reportofthe NINCDS-ADRDAWorkgrouponAlzheimer’sDisease,24.,pp.939–944.

McKhann,G.M.,etal.,2011.ThediagnosisofdementiaduetoAlzheimer’sdisease:recommendationsfromtheNationalInstituteonAging- Alzheimer’sAssociationworkgroupsondiagnosticguidelinesforAlzheimer’sdisease.Alzheimer’sDement.7(3),263–269.

Morris,J.C.,1993.Theclinicaldementiarating(CDR):currentversionandscoringrules.Neurology43,2412b–2414b.

Nelwamondo,F.V.,Mahola,U.,Marwola,T.,2006.Multi-scalefractaldimensionforspeakeridentificationsystems.In:Proceedingsofthe8th WSEASInternationalConferenceonAutomaticControl,Modeling&Simulation,pp.81–86.

Ouayoun,M.,Péan,V.,Meyer,B.,Chouard,C.H.,1999Mar.Astudyofspeechfractaldimensions.ActaOtolaryngol.119(2),261–266.

Péan,V.,Ouayoun,M.,Fugain,C.,Meyer,B.,Chouard,C.H.,2000.Afractalapproachtonormalandpathologicalvoices.ActaOtolaryngol.120 (2),222–224.

Péan,V.M.,Ouayoun,C.,Fugain,B.,Meyer,C.H.,2002.Chouardafractalapproachofpathologicalvoices,fractalsinbiologyandmedicine.Math.

Biosci.Interact.,147–151,http://dx.doi.org/10.1007/978-3-0348-8119-716.

Picard,R.,Cook,D.,1984.Cross-validationofregressionmodels.J.Am.Stat.Assoc.79(387),575–583,http://dx.doi.org/10.2307/2288403(JSTOR 2288403).

Pickover,C.A.,Khorasani,A.,1986.Fractalcharacterizationofspeechwaveformgraphs.Comp.Graph.10(1),51–61.

Pitsikalis,V.,Maragos,P.,2009.Analysisandclassificationofspeechsignalsbygeneralizedfractaldimensionfeatures.SpeechCommun.51(12), 1206–1223.

Praat.DoingPhoneticsbyComputer.Availablefrom:http://www.fon.hum.uva.nl/praat

Solé,J.,Zaiats,V.,2003.Anon-linearVADfornoisyenvironment.Cogn.Comput.2(3),191–198.

Tsonis,A., 2007. Reconstructing dynamics from observables: the issue ofthe delay parameter revisited.Int. J. Bifurc.Chaos 17, 4229, http://dx.doi.org/10.1142/S0218127407019913.

VAD,VoiceActivityDetectoralgorithm.Availablefrom:http://www.mathwork.com

VandePole,L.A.,etal.,2005.TheeffectsofageandAlzheimer’sdiseaseonhippocampalvolumes,aMRIstudy.Alzheimer’sDement.1(1Supp.

1),51.

WEKA.Availablefrom:http://www.cs.waikato.ac.nz/ml/weka/

KarmeleLópez-de-Ipi ˜nareceivedthePh.D.degreeinComputerSciencein2003,andaMasterdegreeinElectronics andAutomationandtheB.Sc.degreeinPhysicsin1990,attheUniversidaddelPaísVasco/EuskalHerrikoUnibertsitatea (UPV/EHU).Sheworkedforenterprisesuntil1995whenjoinedtheDepartmentofSystemsEngineeringandAutomation oftheUniversityoftheBasqueCountry.HewasDirectorattheUPV/EHU(2004–2009).Sheiscurrentlyheadofthe EngineeringandSocietyresearchgroup,ELEKIN.HerresearchinterestsareinBioengineeringandBiomedicalEngineering, PatternRecognition,SignalProcessing,AmbientIntelligenceandRobotics.

JordiSolé-CasalsreceivedthePh.D.degreewithEuropeanlabelin2000,andtheB.Sc.degreeinTelecommunicationsin 1995,bothfromthePolytechnicUniversityofCatalonia(UPC),Barcelona.In1994hejoinedtheDepartmentofDigital TechnologiesandInformationoftheUniversityofVic,wherehewastheDirector(2010–2012).HeiscurrentlyHeadof theDataandSignalProcessingResearchGroup.HeisvisitingscientistwiththeGIPSA-Lab(Grenoble,France),LABSP RIKEN(Tokyo,Japan)andBMU(Cambridge,UK).Hisresearchinterestsareinbiomedicalsignalprocessing(EEG,fMRI, speech,biometricapplications),neuralnetworks,sourceseparationandindependentcomponentanalysis.