using multiple/massive sensors Clustering for filtering: Multi-object detection and estimation Information Sciences

ContentslistsavailableatScienceDirect

Information Sciences

journalhomepage:www.elsevier.com/locate/ins

Clustering for filtering: Multi-object detection and estimation using multiple/massive sensors

Tiancheng Li

^a,b,∗

, Juan M. Corchado

^a,c

, Shudong Sun

^b

, Javier Bajo

^d

a School of Science, University of Salamanca, Salamanca 37008, Spain

b School of Mechanical Engineering, Northwestern Polytechnical University, Xi’an 710072, China

c Osaka Institute of Technology, Asahi-ku Ohmiya, Osaka 535-8585, Japan

d Department of Artificial Intelligence, Technical University of Madrid, Madrid 28660, Spain.

a rt i c l e i n f o

Article history:

Received 8 April 2016 Revised 7 January 2017 Accepted 9 January 2017 Available online 10 January 2017 Keywords:

Target detection Object recognition Filtering Tracking Sensor fusion

a b s t ra c t

Advanced multi-sensorsystemsareexpectedtocombatthe challengesthatariseinob- jectrecognitionand stateestimationinharshenvironmentswithpoororeven noprior information,whilebringingnew challengesmainlyrelatedtodatafusionand computa- tionalburden.Unliketheprevailing Markov-Bayesframeworkthatisthebasisofalarge varietyofstochasticfiltersandtheapproximate,weproposeaclustering-basedmethodol- ogyformulti-sensormulti-objectdetectionandestimation(MODE),namedclusteringfor filtering(C4F),whichabandonsunrealisticassumptionswithrespecttotheobjects,back- groundand sensors.Rather,basedonclusteranalysisoftheinputmulti-sensordata,the C4Fapproachneedsnopriorknowledgeaboutthelatentobjects(whetherquantityordy- namics),canhandletime-varyinguncertaintiesregardingthebackgroundandsensorssuch asnoises,clutterandmisdetection,anddoessocomputationallyfast.Thisoffersaninher- entlyrobustandcomputationallyefficientalternativetoconventionalMarkov–Bayesfilters fordealingwiththescenariowithlittlepriorknowledgebutrichobservationdata.Sim- ulationsbasedonrepresentativescenarios ofbothcompleteandlittle priorinformation havedemonstratedthesuperiorityofourC4Fapproach.

© 2017ElsevierInc.Allrightsreserved.

1. Introduction

Multipleobject/targetdetectionandestimation(MODE),whichliesatthecoreofmulti-objectrecognitionandtracking, involvesthejointestimationofthenumberoftargetsandtheirstatesfromnoisyobservationsthatarereceivedatdiscrete time instants inthe presenceof clutter/false-alarms andmisdetection.The standard andprevailing solution,whetherfor a singleobjectormultiple objects,isbased onthe Markov–Bayesframework, whichemploys a hiddenMarkovmodelto describetheobject’sdynamics,andanobservationfunctiontolinkobservedvariablestothehiddenstates,namelythestate spacemodel(SSM).ThisSSMformulation,whichfunctionsbyfindingaMarkov–Bayesfilter,hasbeenwelldemonstratedin thefieldfordecadessincethepioneeringworkoftheKalmanfilter.Ithasbeenfortyyearssincethefirstsurveyofmulti- objecttrackingmethods[2]andtherelatedtopicsareattractingmoreattentionthaneverwiththeintroductionofthefinite setstatistics[30].

∗ Corresponding author at: School of Science, University of Salamanca, Salamanca 37008, Spain.

E-mail addresses: [email protected] , [email protected] (T. Li), [email protected] (J.M. Corchado), [email protected] (S. Sun), jbajo@fi.upm.es (J. Bajo).

http://dx.doi.org/10.1016/j.ins.2017.01.028 0020-0255/© 2017 Elsevier Inc. All rights reserved.

The concept of “simultaneous signal detection andestimation” appeared since as early as[34] where detection was treatedasaspecial caseofestimation,orestimationasageneralizeddetectionprocess.The primarychallenge forMODE involvesawide rangeoflatentstatisticalmodels/uncertaintiesregardingobjectquantity anddynamics(including appear- ance,movement,deformationanddisappearance),sensorprofiles(suchasirregularrevisitfrequency,misdetectionandout- of-sequencedata)andbackground(suchasclutter); simply,realworld noisescanbe distorted,correlated,coloredand/or multiplicativebuthardlybe additiveGaussianwhichisrequiredbyKalmanfilters.Mostofthetime,thesestatisticalmod- elsareunknownandcanonlybeapproximatedonthebasisofonlineorofflinedata[35],namelysystemmodeling(which isalso referred toas systemidentification orthe designof theprior [43]). Evenifthey can be accurately established to a sufficientextent,thefull Bayesian,oreven onlythenecessarylikelihood[16,33,37,44],computationcould beprohibitively intractableandapproximationmustberesortedto.

Morespecifically, theinstability and inconsistencyofthe Bayesian inferencefor eitherlinearor nonlinearmodels has beenrecognized [15,17]. It hasbeendemonstrated that theuse ofa stochastic filtercould be inefficientorcounteractive incertain cases[23,36].Recently, we proposed the concept of probabilityof filter benefit (PoFB)[23] based on the data- drivenobservation-only(O₂)inferencetoclarifythisissue:itisonlywhenthePoFBislargerthan0.5thatthefilterismore preferablecomparedtotheO₂inference.

Despiteimmense ongoing efforts spent ondeveloping newMarkov–Bayes filters,MODE in aharsh environment with littleprior informationremains anopenchallenge,regardlessoftherealtimecomputingburden.Nowadays,theadventof multiple/massivesensorsystems providesvery rich observationathighfrequencyyetlow financialcost. Thisfacilitatesa novelperspectivebasedonsensordataclusteringtodealwithfalse alarmsandmisdetection, namedclusteringforfiltering (C4F) inthis work, withoutrelying onunrealistic statistical assumptions aboutthe objects, sensors andthe background.

Theproposedmulti-sensorC4Fmethodologyhascomparablybetter‘frequentist’interpretation,andisexpectedtoproveits mettleinthechallengingscenariowithlittlepriorinformation.

The paper is organized as follows. Section 2 summarizes the motivation and outline of our methodology. Section 3presentstheideaofthemulti-sensorO₂inferenceinwhichtypicalsensormodelsusedforobjectrecognitionandtracking arealsodiscussed.Section4presentstheC4Fmethodologyindetail,whichconstitutesthecorecontributionofthispaper.

Section5presentssimulationdemonstrationsincludingacomparisonwithstate-of-the-artalgorithms.Section6concludes thepaperanddrawsattentiontoourfuturework.

2. Motivationandoutline

TheSSMdesignforMarkov–Bayesinferenceinvolveschoosingadynamicalfunctiontodescribethestateevolvingprocess overtime,andanobservationfunctiontolinkobservedvariablestothehiddenstates.Hereafter,theuncertaintiesaboutthe objectquantity anddynamics, thebackground andsensorprofiles (e.g. detectionuncertaintyandnoise) all ofwhichare inherentlyvagueinreality,arejointlyreferredtoastheStatisticalModelinformation,whichexcludesthedeterministicpart oftheobservationfunction thatis preciselygivenapriori. Whendesigning optimalestimatorsit isunrealistictoassume thatthe statisticalmodelisknownperfectly.The issueisthen todesigna robust estimatorthatis optimalrelativetoan uncertaintyclassofprocesses[6].Effortshavebeendevotedfromvariousaspects,includingthefollowing,tonameafew:

• Non-cooperative/intractableobjectmotion:constrained[28,35,41]/sparse[1]/circular[19]state sequenceandmaneuvers [14,27,43],etc.,

• Unknown backgroundandsensoruncertainties:noisecharacteristics[8],detectionprobability[13],clutterrateandde- tectionprofiles[31,46,49],network-induceddelay[7],etc.,and

• Imperfectdata:asynchronoussensordata[14,42],intractablelikelihood[16,33,37,44],measurementrandomlatency[48], outlier[32]andcontinuous-timeobservation[12],etc.

Giventhatweareparticularlyinterestedinthemean-squareerror(MSE)oftheestimation,theoptimalestimatoristhe leastmeansquares(LMS,alsoreferredtoasminimumMSE,MMSE)estimator,whichcalculatestheconditionalexpectation ofthestatextgivenallthecollectedobservationsz_1:ttothenewesttimetasfollows:

ˆ

xt=E

(

^xt

|

^z1:t

)

=∫xp

(

^x

|

^z1:t

)

^{dx (1)}

wheretheconditional/posterior distributionp(xt|z_1:t)consistsofthepriorinformation(relatedtothestatetransitionfunc- tion)andthelikelihood (relatedtotheobservationuncertainty), bothofwhichdepend onthestatisticalmodelsover the timeseriesm_1:t.

When thestatistical modelis not givena priori,a generalBayesian solution shallestimate the statisticalmodel m_1:t jointlywiththestatex_t (alsoreferredtoashierarchicalBayesianmodeling),i.e.,

p

(

^xt,m1:t

|

^z1:t

)

^=p

(

^xt

|

^z1:t,m1:t

)

^p

(

^m1:t

|

^z1:t

)

⁽²⁾

GivenamodelsetMconsistingofasufficientnumberofcandidatemodels,thefullprobability formulaandtheBayes’

ruleshowus

p

(

^xt

|

^z1:t

)

^{= }

∀ⁱ∈[1,t]:mi∈M

p

(

^xt

|

^z1:t,m1:t

)

^p

(

^m1:t

|

^z1:t

)

= 1

p

(

^z1:t

)



∀ⁱ∈[1,t]:mi∈M

p

(

^xt

|

^z1:t,m1:t

)

^p

(

^z1:t

|

^m1:t

)

^p

(

^m1:t

)

where p

(

^z1:t

)

= 

∀ⁱ∈[1,t]:mi∈M

p

(

^z1:t

|

^m1:t

)

^p

(

^m1:t

)

. (3)

Clearly,themodelsetMshallbecompletetocontainallsignificantpotentialcandidates(ifnotimpossible),whichhow- everwillcauseremarkablecomputingrequirements.Ausefulstrategyinthisaspectisto adaptivelyinteractbetweenthe candidatemodelsassumedinthesetMwhichcansavetherequirednumberofparallelfilters,suchastheso-calledinter- actingmultiplemodel(IMM)[27].Still,therealsystemmaychangeso frequentlythat thealgorithmcanhardly catchup, oreventhebestmodelsmaybeinadequateforrepresentingkeyfeaturesoftherealsystem,sufferingfromerroneousfrac- tions.Thesituationwillbecomemuchworseinthecaseoftrackinganunknownandtime-varyingnumberofmaneuvering objects byusing heterogeneoussensors. Inanycase, theperformance of theMarkov–Bayesestimator dependsgreatlyon thecoincidingdegreebetweentherealsystemandthemodelassumed[27,45].

Comparedwithsingleobjectdetectionandestimation(SODE),clutterandmisdetectionwillsignificantlyaffecttheobject detection(whichisrelatedtodeterminingthenumberofobjects).Eveninanidealsituationwherethedynamicmodelsare established andobjects are detected perfectly,the correspondencebetweentheobservations andthe objectsin theclut- tered environment isoften ambiguousandposes another fundamentalchallenge. The wayinwhich thiscorrespondence isidentified distinguishestwo maingroupsofexisting solutions.First,conventionalsolutions decomposetheMODE prob- lemintomultipleSODEproblemsbasedonobservation-to-object association,whichisinherentlyabletoprovidecomplete

‘track’information.Oneofthemainobstacles inthisaspectisthat thenumberofpossibleassociationswillexponentially increase over time andmust bemerged orpruned atthe expenseof reducing precision forrealtime filtering. Secondly, solutions based on finiteset statistics such as the probability hypothesis density(PHD) filters andmulti-Bernoulli filters [30,31,46]provide anapproximation ofthemulti-objectBayesian recursionwithoutexplicitobservation-to-object associa- tion.Inbothcases,approximationisindispensableforbothmodelingandcomputinginnonlinearsystems.

Ontheotherhand,therapiddevelopmentandjointdeploymentofsensors(withagraduallyimprovedrevisitfrequency andaccuracybutalowercost)facilitatecombatingeitherthepoorpriorknowledgeorindividualsensorfailures,providing richer andmore reliable observation[5,18].For instance, an ordinary smartphone encompassesa variety of sensors that permit multi-sourceWIFI, Bluetooth, GPSsignals aswell asaltitude,acceleration,directionandeven contextinformation based on embeddedsoftware,which can all be explored for the sametaskof “mobile positioning”. While the hardware strengthwillincreasethecapacityofthesystem,leadingtoafavorableadvantageformoreaccurateestimation,itwillalso posenewchallengesforoptimaldatafusionandrealtimecomputation.Itisworthnotingthat,themoreaccuratethedata are,themoresensitivetheestimatoristomodelerrors[23].

Givenalltheproblems,challengesandopportunitiesformulti/massive-sensorMODEinharshenvironmentswithlittle priorinformation,aninterestingquestioniswhetherwehaveotherefficientsolutionsthatcanavoidtheneedtodesigna sophisticated,computationallyintensiveandmodel-sensitivefilterinordertomaximallymeettherequirementforreliable andrealtimeMODE.Thisformsthestartingpointofourwork,bywhichweadvocatetheuseofclusteringtechnologiesfor filtering,namedC4F.ThepresentC4Fsolutioncomprisesthreemajorsteps:

1) Project all (synchronous) sensor observations from the observation space to the state space, to generate multiple detectionsthatcorrespondtorealobjectsandfalsealarms;thedetailisgiveninSections3.1–3.2.

2) Clusterthedetectionsaccordingtotheirspatialdistributiondensityandcertainconstraints:thosecongregating/clus- teringcorrespondtothesameorcloseobject(s)whilethosethatarescatteredwillbeidentifiedasfalsealarms(and discarded);thedetailisgiveninSections4.1–4.2.

3) Thedetectionsthatareidentifiedasbelongingtothesameobjectwillbe fusedinthemannerofleastsquaresesti- mation,whichformsafinalstate-estimateofthatobject;thedetailisgiveninSection4.3.

ThepresentmultiplesensorC4Fapproachdiffers fromexistingfilter-basedtargetdetectionalgorithmsbyreleasingthe needtomakestatisticalassumptionsabouttheobjectquantityanddynamics,sensordetectionprofilesandevenobservation noises.Wenoticedafew worksthatemployclusteringwithinfilters,e.g.,[20]andvarious adaptive/robustestimatorsthat have beenproposed to accommodate little/poorprior information; however,their core isa model-specific filter.While a limitedpartofthisworkappearedearlierin[21]foridenticallydistributedsensorsonly,newanddetailedcontentwillbe presentedinthispaper.

3. Multi-sensorO₂inference

Theconcernedmulti-sensorMODEscenarioisgeneral,whichcanbedescribedasfollows:anunknownandtime-varying numberofobjectsevolveintermsofbirth,disappearance,deformation(includingmergingandsplitting)andmaneuvering

bothinstatespaceandintime.Anumberofconditionallyindependentsensorsareusedforobservingthescenario,whose observationfunctionandpositionsareknown.Theobjectsmaybedetectedormissedbythesensorswithunknownprob- abilities,perhapsatirregularobservationrates,inthepresenceoflatentandtime-varyingsignificancelevelsofclutterand noises.Toavoiddeepertechnicalissuesthatdetractfromthecoreideaofourwork,somecommonassumptionsaremade totheobjects,clutterandsensors,respectively:

(A.1) Eachobjectgeneratesobservationsindependentlyoftheothers,andone objectgenerates nomorethanoneobser- vation ateach sensorateach scan.In other words,we contendwithonlythepoint-object (regardlessof extended objectormultiple-observationduetoreflection)andneitherconvoysnorengagementsareinvolved.

(A.2) Theclutterwillnotcongregate orclusterintheview fieldofeachsensor.Thisassumptionisrestrictivealbeitcom- monintheliteraturesinceinspecialcasescluttercancongregate incloselyplacedsensors.Forinstance,spikesdue to theseasurfacereturns arelikely tobe detected bymultiple maritimeradars;however, wewill notaccount for trickycaseslikethisatthepresenttime.

(A.3) Allsensors workindependently,synchronouslyandtheaverageobjectdetectionprobability theyprovideisnot too low,otherwisetrackingbeforedetection(TBD)isadvised.

3.1. Projectingobservationstothestatespace

Inthegeneralformulation,thesensorperformsperiodicscansbasedonaknownobservationfunctionht(·)

zt=ht

(

^xt,

v

t

)

⁽⁴⁾

wheretindicatestheobservationtime−instant, x_tdenotesthestate, z_ttheobservation, andv_ttheobservationnoise.

Astheconcernofthispaper,nostatisticalinformationisfeasibleaboutthetargetmotionandtheclutter.Weresortto thepuresensordatafordetectionandestimation,namelyO₂ inference.Itcanbeconceptuallywrittenasfollows(aslong astheobservationfunctionisgenerallyinvertible):

χ

ˆ^t=ht⁻¹

(

^zt,

v

^¯t

)

⁽⁵⁾

whereh⁻¹_t isthe“generalized” inversingfunctionofhtinrealvariablespace,

χ

ˆ^{tistheinferenceof“theobservedcomponent} of” thestatextand

v

¯t isanaverageusedtocompensatetheobservationerrorandcanbespecifiedasthemeanofthenoise distributionE(v_t).Inafullyobservedsystem,

χ

ˆt isafull-dimensionalstateestimate xˆ_t whichshouldbe thebestfitofthe observationdata,i.e.,

xt=argmin

x



^zt− ht



x,

v

t

 

⁽⁶⁾

orinatimeseriesmanner,

xt1:t2=argmin

xt1:t2 t2



t=t1



^zt− ht



xt,

v

t





⁽⁷⁾

where



^x−y



^{isameasureofthedistancetobedefinedbetweenxandy,[t}1,t₂]isthetimewindowunderconsideration andthesummationisoverallthesamplinginstantsinthetimewindow.

However,ifanytargetmotionmodel informationisgivenaprioriorcanbe learnedfromtheonlinesensordata(e.g., thestateevolvingisastationaryprocess),suchasthedeterministicpartofthestatetransition[17],stateconstraints[47], modelerror[3]orevenonlya context-informationlike“trajectory issmooth”[25],itshouldbetakenintoaccountinthe formulationof(7),tobeaddressedinSection6.

Whenthestatisticsofthesensornoiseisavailable,theMonteCarlo (MC)inversingapproachcanbeappliedtoaccom- modatenonlinearobservationfunctionhtforunbiasedestimation,forwhicha setofsamplesarerandomlysampledfrom thenoisedistribution

v

⁽_tⁱ⁾∼

v

t,i=1,2,...I inordertocarryouttheinversingcalculationof(5),i.e.,fori=1,2,…I

χ

ˆt⁽ⁱ⁾=h⁻¹_t



zt,

v

_t⁽ⁱ⁾



(8) Then,itisstraightforwardtocalculatetheunbiasedestimateanditscovarianceasfollows

χ

ˆ^{t= 1}_I ^I

i=1

χ

ˆ⁽tⁱ⁾ (9)

Cov

 χ

ˆ^t



= 1 I− 1

I

i=1



χ

ˆt⁽ⁱ⁾− ˆ

χ

^t



χ

ˆ⁽tⁱ⁾− ˆ

χ

^t



T

(10)

Tospeedupthecomputation,thesamplescanbecreatedinadeterministicmannersuchassigma-points.Wemustnote thatthesensoruncertaintycannot beexactly knownintherealworld,notto saythatonlysufferingfromasinglewhite noise.Ifthestatisticsofvtisunknown,anapproximateerrorsuchasthatcausedbyomittingthenoisewillnotaffectthe O₂ inferenceasmuchasdoestothesequentialBayesianfilters.

Morechallenging,theobservationfunctionmaybemathematicallyirreversibleandthereforepreventsthedirectinvers- ing calculation.Practicaltreatments tothisissue,referredto asdynamicinverseproblem[38],canbe found.However,in thecontextofobjecttracking,sensorsarelimitedtoafewtypesandthejointuseofmultipleorevenmassivesensorscan improvetheobservabilitytocircumventtheirreversibility.Inthecaseofmassivesensors,itwouldbeinterestingtofindan optimalobservationconfigurationbyproperlygroupingthesensorsbasedontheirspatialdistribution(suchasminimizing the squaredpositionerror bound [40]) andoptimallyfusing their results.In thispaper,we concentrate onthe casethat eachsensorisasufficientsensoryunitforsimplicity.

3.2. O₂inferenceontypicaltrackingsensors

WeconsiderthreerepresentativesensorsforobservationinthetwodimensionalCartesiancoordinate.

Positionobservation (e.g.,visualcamera): theobservationis directlymadeonthe planarposition [p_x,t, p_y,t]^T ofthe object,givenby

zt=



z_x,t zy,t

=



p_x,t py,t

+

v

t (11)

where[z_x,t,z_y,t]^Tistheobservedpositionoftheobjectinxandydimensions,respectively.

Asaddressed,theunbiasedO₂ inferenceisgivenby



pˆ_x,t pˆy,t

=



z_x,t zy,t

− E

( v

t

)

⁽¹²⁾

Thevarianceoftheestimateisequivalenttothatoftheobservationnoisevt.

Range-bearingobservation(activeradar):theobservationisanoisyrangeandbearingvector,givenby

zt=



rt

θ

^t

=



(

^px,t− Sx,t

)

²+

(

^py,t− Sy,t

)

² arctan



py,t−Sy,t

px,t−Sx,t

 

+

v

t (13)

where[px,t,py,t]^Tand[Sx,t,Sy,t]^Tarethepositionoftheobjectandofthesensor,respectively.

ToimplementtheO₂inference,oneestimatecanbeinferredbyinversing(13)aftertakingoff vt,yielding



pˆx,t

pˆ_y,t

=+/−

⎡

⎣



r_t² 1+θt²

tan

( θ

t

) 

r_t² 1+θt²

⎤

⎦

+



Sx,t

S_y,t

(14)

Asshown,inversingthearctanfunctioninvolvesasignproblem,denotedby¨+/−¨ in(14).Often,thestateisconstrainted ineitherpositiveornegativestate spaceandthesignproblemisavoided; seeoursimulationinSection5.1.Otherwiseat leasttwoactivesensorsplacedatdifferentpositionsareneededtoinferthesignbytriangulation.

Asstated, theestimategivenby(14) bysimplyabandoningthenoiseisactuallybiasedevenifE(vt)=0.Ifthestatistics ofthenoisev_t=[v_1,t,v_2,t]^Tisknown,theMCdebiasingschemecanbeapplied,i.e.,samplingasetofsamplesaccordingto thenoisedistribution[

v

⁽₁ⁱ_,t⁾,

v

₂⁽ⁱ_,t⁾]^T∼ p(

v

t),i=1,2,...I,yieldingsimulatingestimatesasfollows:

pˆ⁽_xⁱ_,t⁾ ˆ p⁽_yⁱ_,t⁾



=+/−

⎡

⎢ ⎢

⎢ ⎢

⎢ ⎢

⎣

 

 

^{rt−v(1i,t)}

2

1+ θ^t−v⁽2ⁱ,t⁾

2

tan



θ

^t−

v

₂⁽ⁱ_,t⁾

    

^{rt−v(1i,t)}

2

1+ θt−v₂⁽ⁱ_,t⁾2

⎤

⎥ ⎥

⎥ ⎥

⎥ ⎥

⎦

+



_S

x,t

Sy,t

(15)

Themeanandvarianceoftheestimate(foreachobject)canbeobtainedrespectivelyby



_pˆ

x,t

pˆ_y,t

=

⎡

⎢ ⎣

1 I

I i=1

ˆ p⁽_xⁱ_,t⁾

1 I

I i=1

ˆ p⁽_yⁱ_,t⁾

⎤

⎥ ⎦

⁽¹⁶⁾

Var



ˆ px,t



Var



ˆ py,t





=

⎡

⎢ ⎣

I−11

I i=1



ˆ p⁽_xⁱ_,t⁾− ˆpx,t



2

1 I−1

I i=1



ˆ p⁽_yⁱ_,t⁾− ˆpy,t



2

⎤

⎥ ⎦

⁽¹⁷⁾

Bearing-onlyobservation(passiveradar):theobservationisbearingonly,givenby

θ

^t=actan



py,t− Sy,t

px,t− Sx,t



+

v

t (18)

Toimprovetheobservability,twobearingsensorslocatedatdifferentpositionscanbecoordinatedtoinfertheposition ofobjectsbytriangulation.Assumingthattheyarelocatedatpositions[S_x1,t,S_y1,t]^T and[S_x2,t,S_y2,t]^T andreportbearing

θ

1,t

and

θ

2,t,respectively,theestimate[pˆx,t,pˆy,t]^Tcanbecalculatedbysolvingthefollowingequationset

⎧ ⎨

⎩

θ

1,t=actan



pˆy,t−Sy1,t ˆ px,t−Sx1,t

 θ

2,t=actan



_pˆ

y,t−Sy2,t ˆ p_x,t−Sx2,t



⁽¹⁹⁾

whichgives(ignoringthesignproblemherebytreatingitaspositive)

⎧ ⎨

⎩

ˆ

p_x,t=^{tanθ1Sx1,t−tan}_tanθ1^θ_−tan^2Sx2,t_θ⁺₂^{Sy2,t−Sy1,t} ˆ

py,t=^tanθ1

(

^{tanθ2Sx1,t−tanθ2Sx2,t+Sy2,t−Sy1,t}

)

tanθ1−tanθ2 +Sy1,t

(20)

Toguaranteethat(20)uniquely exists,neither[Sx1,t,Sy1,t]^T and[Sx2,t, Sy2,t]^T nor

θ

1,t and

θ

2,t can bethesame,namely theequationsetneedstobeconsistent.Asstated,ifthenoisevtisexactlygiven,debiasingshallbeapplied.

Likewise,iftheobservationisonlyarange(orequivalent,suchasrange-rate[50]),atleasttwosensorsplacedatdifferent positionsarerequiredtoinfertheplanarposition/velocity,whichisomittedhere.Infact,theideaofprojectingthesensor datatothestatespacehasbeeninvolvedinwirelesstriangulation,trilaterationandmultilaterationpositioningtechnologies basedonangleofarrival,signalstrengthobservationsandtimedifferenceofarrivalinformation,respectively;seee.g.,[29]. 3.3.Observability

One active range-bearing sensor or one position-observed sensor suffices to conduct O₂ inference and is referred to asasufficient sensorysensor; todo so,atleasttwo range-onlyorbearing-onlysensorslocated atdifferentpositions are requiredtowork jointlyforpositionestimation.We willnot emphasizethisissuehereasa sufficientnumberof sensors areusedinourcaseandtheobservabilityisalwaysstronglyguaranteed.Thechallengehereisnotunder-observabilitybut over-observability.

Clearly,the basicO₂ approach onlyestimatesthe dimensionsofthe statethat have beenobserved.Note thateven in filterstheunobserved dimensionsareinferred fromtheobserved dimensionsbased onthephysical lawcontainedinthe Markovtransition matrix,e.g., the differentiation ofthe position over time is the velocity and the differentiationof the velocityistheacceleration.Therefore,basedonsuccessiveestimatesofpositionorvelocity,wecaninferonefromtheother.

Whenfurthertargetmotioninformationisavailable,suchas“nearlyconstantvelocity”,theestimatescanbecoordinatedby fittingthemovertime-seriesandtheiraccuracybeimproved[25].Inthispaper,weareonlyinterestedinobjectdetection andpositionestimation.

4. Multi-sensordataclusteringforMODE

Asassumed in(A.1–3), all sensors work synchronously,reporting massiveconditionally independent detectionsin the observationspace.RealizingtheO2inferenceonthesedetectionswillgeneratecorrespondingpotentialestimates(alsocalled detectionsinthefollowing) inthestate space, consistingoftherealestimatesofobjects andfalse alarms becauseofthe clutter.Inthe sequence,we willpresenta constrainedclusteringapproach todistinguish thereal estimatesofindividual objectsfromeachotherandfromthefalsealarms,whichisthecoretaskrequiredforMODEascomparedwithSODE,and formsanovelperspectivefor“decluttering”.

4.1. Expectedclustersize

Theclustersizeisgivenbythenumberofdetectionsinthecluster,whichdependsonboththenumberofsensorswhose field of view (FoV) covers that cluster, and their respective detection probabilities. In mostcases, the sensor’s detection capacityisstate-sensitiveandtherefore,we denotethe objectdetectionprobability of sensorsat areaa asp_D,s(a) which issmallerthan 1.Denoting theset ofall sensorsasSandtheset ofsensors whoseFoVcovers areaa asS_a,theexpected numberofsensorsthatdetectanobjectexistingatareaaisgivenby

E

(

^na

)

^=_s∈S

a

pD,s

(

^a

)

^

|

^Sa

|

⁽²¹⁾

wherenaisthe numberofdetections reported(forasingleobject)atarea a,|Sa| givesthenumberofelementsinsetSa

and“࣠” means“smallerthanorapproximatelyequalto” inwhich“approximatelyequalto” isbecausecluttermayfallin thecluster.

Algorithm 1 Multi-sensor data clustering.

Step 1 . Project all sensor observations into the same Cartesian coordinate based on the O 2 inference, obtaining undistinguished detections corresponding to either real objects or false alarms; see Section 3 .

Step 2 . Detections from different sensors will be identified as connected if they satisfy ( 22 ) and ( 24 ).

Step 3 . Calculate the number of detections in each cluster and identify whether the cluster is over-sized via ( 25 ). If k a ≥ 2, cluster C a will be further partitioned into k a sub-clusters conditioned on (21–22).

4.2. Declutteringviaclustering

Basedon(A1–3), thedetectionsofasingleobjectgivenby differentsensors willclusterlocally,whilefalsealarmswill not.Consequently,wehavethefollowingcriteriontodistinguishtherealobjectsfromfalsealarms:

Criterion1Thedetectionsareofhighdensityintheareacontainingobjectsandareoflowdensityanywhereelse.

Whileageneraldensity-basedclusteringmethod(suchasDBSCAN[9])thatdistinguishestheareasofhighdatadensity fromlow-densityregions willbe helpfultodistinguish distantobjects,we alsoneed todistinguishbetweencloseobjects (namelyoverlappingclusters)thatsharethesamecluster.Todoso,twoconstraintsshouldbesatisfied:

Constraint1Thedetectionsreportedbythesamesensorcannotbeclusteredintothesamecluster.

Constraint2Thesizeofeachclustershallbedeterminedsuchthat(21)isrespected.

Constraint1isduetothepoint-objectassumption(A.1)andisreferredtoasthe‘cannotlink’(CL)constraint.Denoteall thedetectionsgeneratedinsensorsasasetXs=

{

^xs1,x^s₂,...x^s_ms

}

^wherems=|Xs|isthetotalnumberofdetections.TheCL constraintstatesthat

c ₌



x^s_i,x^s_j



,

∀

^{i =j (22)}

where c ₌(^xs_i,x^s_j) ^{means that xs}_i,x^s_j cannot belong to thesame cluster/object. Tothis end, their distance



^xs_i− x^s_j



^{can be}

treatedas infinitein thedistance-based clusteringalgorithm design.In the Cartesian coordinateforx^s_i=[p^s_x,i,p^s_y,i]^T,x^m_j = [p^m_{x, j},p^m_{y, j}]^T,the2 normEuclideandistance



^xsi− x^m_j



2isgivenby



^xsi− x^m_j



2:=



x^s_i− x^m_j



T



x^s_i− x^m_j



=

 

p^s_x,i− p^m_{x, j}



2

+



p^s_y,i− p^m_{y, j}



2

(23) Theproposed constraineddensity-based clusteringmethodcanbe describedasshowninAlgorithm1.InStep2ofthe algorithm,a ‘neighbordistance’threshold(which resemblestheparameter “neighborradius

ε

^{” usedinDBSCANandgives}

the maximum distancebetween two detections fromdifferent sensors fortheir directconnection to be included in one cluster)isrequiredtoassociatedetectionsfromdifferentsensorsthatverylikelycorrespondtothesameobject.Oursolution is to determine it accordingto the covariance ofthe distribution of all detections forthat object,reflecting the average observationaccuracyofallsensors.Consideringthemostcommoncaseofellipsoidalclustershape,we proposetousethe ruleofellipsoidalvalidation,i.e.,twodetectionsx^s_i,x^m_j areconsideredtobeincludedinthesameclusteriftheysatisfy



x^s_i− x^m_j



T



Cov



ˆ xt



−1



x^s_i− x^m_j



≤ l1 (24)

whereCov(^xˆt)^{isthecovarianceoftheestimationgivenbytheproposedMCsamplingmethodasshownin(10),onaverage} overallsensorsandl₁isthefirstparameterneededinourapproach.Wesuggestascalingscopel₁∈[1,4].

Whenmultipleobjectsappearclosely,theirdetectionscanbeeasilyclusteredintoonecluster.Therefore,topartitionthe over-sizedcluster,anotherthreshold

ρ

^{isneededtogivetheaveragenumberofdetectionsina}single-objectcluster.Itwill bedesignedwithrespecttotheexpectednumberE(na)ofdetectionsforasingleobjectatlocal,e.g.,

ρ

^a=l₂×E(na),where 0< l2 < 1isscalableandinvolves atrade-off betweenmissingdetections andcausingfalse alarms.We recommendl2 ∈ [0.6, 0.9].AccordingtoConstraint2,thenumberkaofsub-clusterscontainedinclusterCasatisfies

ka

ρ

^a≤

|

^Ca

|

^<

(

^ka+1

) ρ

^{a (25)}

Giventhat thenumberofsub-clustersisdeterminedby(25),theover-sizedclustercanbetheneasilypartitionedsince falsealarmisnolongerinvolvedandthestandardk-meansclusteringalgorithmisapplicable.Tonote,eachsub-clustershall haveapproximately

ρ

^{abutnomorethan|Sa|detectionswhilesatisfying(22).}

Remark 1. Scaling parametersl₁ andl₂ implyconfidence levels assignedto coordinatingthe neighbordistance

ε

^andthe

cluster size

ρ

^{, with referenceto the sensors’} observation noise variance and detection probabilities. They are essentially correlated, asa largerl₁ indicates more detections to be associated ineach cluster and correspondingly a largerl₂ to be used.

Remark2. Inthecaseofextended/groupobjects,thecluster canbeofnon-ellipsoidalshapes, forwhichextraconstraints havetobe madeonthecluster shapeandtakenintoaccountforfinerpartitioningoftheoverlappedclusterinStep3 of Algorithm1.

Inthispaper,weassumethesensors’profilesareeitherknownoreasytoapproximateinordertofacilitatethecalcula- tionof(21)and(24),whichisindeedafairassumptioncommonlymade.Aslongasnottoomanyobjectssimultaneously

−2500 −2000 −1500 −1000 −5000 0 500 1000 1500 2000 2500 200

400 600 800 1000 1200 1400 1600 1800 2000

x (m)

y (m)

Fig. 1. Real object-positions (red “x”), detections (green ‘.’), clustering estimates (black “o”) and FoV bounds (blue circle) of 10 sensors (located at ‘  ’). (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article).

appearatthesameplace(which arehard todifferentiate), aslightchangeofl₁orl₂will notaffecttheresultmuch.This addsto theflexibility and robustnessofour approach,which accommodatesinaccurate knowledge ofthe sensors andis littlesensitivetotheaccuracyoftheparameters.

Moreover,adaptiveonlinelearningprocedureswereproposedtodealwithunknown sensorprofilesincludingthenoise statisticsanddetectionprobability in[22],in whichtheneighbor distance

ε

^{andthecluster size}

ρ

^{can notbe calculated}

directlyasaddressed,butmustbelearnedfromthesensordata.Tothisend,theaveragenumberofdetectionsthatoriginate fromthesamesensorintheover-sizedclusterwasusedtoapproximatethesingle-targetclustersize.Then,insteadof(25), forover-sizedclusterCa,thenumberofsub-clusterscanbegivenas

ka≈

1

|

^Sa

|



s∈Sa

|

^Xs∩Ca

| 

(26)

where [·] givesthenearest integertothe content.Clearly,a larger|S_a|indicates a betterestimation.Giventhatonly one objectexistsinclusterCa,wehaveE(

|

^Xs∩Ca)=1.

Forillustrationpurposes,an exampleisgiveninFig. 1forone scenarioofthe simulationconductedinSection 5.1.In thisexample,10homogeneousrange-bearingsensors(markedas‘’)ofdifferentFoVs(theirboundsaregiveninindividual bluecircles) are placedon ahorizontalline.The detectionsfrom differentsensorsare all projectedinto thesame planar spaceasshownbythegreendotsinthefigure.Usingtheproposed constrainedclusteringmethod(withparametersl₁=2, l₂=0.6), detectionsare clusteredandfused (see the next subsection),obtaining the final estimates(marked byblack ‘o’) whichareapparentlyquiteclosetothetruepositionofthesixobjects(marked byred‘x’). Inparticular,closeobjectsare correctlydistinguishedfromeachotherandthenumberofobjectsiscorrectlyestimated.Quantitativeanalysiswillbegiven inSection5.1.

4.3.MMSEclusterfusion

Giventhat thereal object-detectionsare distinguishedfromfalse alarms andfromeach other by the givenclustering algorithm,andthattheirvariancesarecalculatedbytheproposedMC samplingapproach,wecannextfusetheserelevant detectionscontainedineachclustertoextractthefinalstate-estimateforeachobject.

Denotingthemeanand(co)varianceofthedetections(whicharenot necessaryGaussian variables)inthesamecluster asm_iandP_i, i=1,…Nrespectively, the bestlinearunbiased estimator(BLUE, alsoreferred toaslinear MMSE estimator) fusestheseconditionallyindependentdetectionsaccordingtotheir(co)varianceas

m_{1,2,...N}=

_N

i=1P⁻¹_i mi

N

i=1P⁻¹_i (27)

IfallthesedetectionsaresubjecttoGaussiandistribution,thefinalestimateisalsoGaussiananditsvarianceis

P_{1,2,...N}=



N i=1

P⁻¹_i



−1

(28)

Furthermore,ifthesesensorsareofthesameaccuracyoverthesurveillancearea(namelythesameobservationnoiseof variance),then(27–28)reduceto

m_{1,2,...N}= 1 N

N

i=1

m_i (29)

P_{1,2,...N}= P

N (30)

Remark3. Eq.(29)isfreeofPindicatingthatthefusioncanbeoptimallycarriedoutwithoutexactlyknowingthestatistics ofthe sensorsaslong asthey areknown tobe ofthesame accuracy.This facilitatesMMSE datafusion ofhomogeneous sensorsofunknownstatistics.

Whenthedetectionisdefinedinconstrained,boundedorcircularstatespaces,constrainedfusionwillbe involved.The readerisreferredto[19,35]andthereferencestherein.ComparedtoaMarkov–Bayesfilter,theproposedC4Fapproachmay notbeprecisewhenfewsensorsoflowqualityareused,butitsaccuracywillbesurelyimprovedwiththeincreaseinthe numberofunbiasedsensorsasshownin(28)/(30).ThishowevermaynotholdinpracticeforaMarkov-model-specificfilter, tobeshowninoursimulationstudynext.

While (parametric) Cramér–Raoand Weiss–Weinstein lower bounds[10] implythe best possible performance (in the senseoflowestmeansquareerror)ofanyestimator,theO₂inferencegivestheworstperformanceofanyeffectiveestimators with respect to a particular observationsequence. In thissense, the C4F approach that extends the O₂ approach to the multi-objectcaseinclutteredenvironmentsthereforeindicatestheworstperformanceofanyeffectiveMODEsolutionswith respect toparticular sequences ofobservations ofmultiplesensors. We willshow next that basedon thisdefinition,the suboptimalfiltersarenotalwayseffective.

5. Simulations

Inthissection,tworepresentativescenariosofrespectivecompleteandlittlepriorinformationareconsidered.Thewell- definedoptimal sub-patternassignment (OSPA) metric[39] isused to evaluate themulti-object estimationaccuracy. For finitesubsetsX={x₁,x₂,…,xm}andY={y₁,y₂,…,yn}wherem,n∈N₀=

{

⁰,1,2,...

}

^{,theOSPAmetricoforderpbetween}

XandYisdefinedas(ifm≤ n) d¯⁽_p^c)

(

^X,Y

)

⁼



1 n



minq∈n

m

i=1

dc



xi,yq(ⁱ)



p

+c^p

(

^{n− m}

)

 

¹p

(31)

wheredc(x, y)=min(c, d(x, y)), the cut off parameter c > 0 and d(x, y) is the Euclidean distance as defined by (23).

d¯⁽_p^c)(^X,Y)=d¯⁽_p^c)(^Y,X)^ifm>nandd¯⁽_p^c)(^X,Y)=0ifm=n=0.

Theorderparameterpdeterminesthesensitivitytooutliers,andthecut-off parametercdeterminestherelativeweight- ing ofthe penaltiesassignedto cardinalityandlocalizationerrors. Clearly,a bettertarget detectioncapacitythat renders moreaccurateestimationofthetargetnumberwillsignificantlyreducetheOSPAmetric.

5.1. Completelyknownbackground

Tosetup thesimulation,the realobjecttrajectoriesare giveninFig.2whereeach startsat‘࢞’ andendsat‘’ (with both theoriginandending timesnotedinthe figure).The objectbirth process issubjecttoPoisson withintensity

γ

^t=

4

i=1r_t,iN(·; Bi,Q)^,whereB1=[−1500,0250, 0,0]^T,B₂=[−250,0,1000,0,0]^T,B₃=[250,0750,0,0]^T,B₄=[1000,0,1500, 0,0]^T,Q=diag([10,10,10,10,

π

^{/180]T)2and r}t,1=rt,2=0.02,r_t,3=r_t,4=0.03.

The objectstatext=[x˜t,wt]^Tconsistsofthe planarpositionandvelocity x˜t=[px,t,p˙x,t,py,t,p˙y,t]^T andtheturnratewt. Eachobjecteithercontinuestoexistwithsurvivalprobability p_S=0.98andmovetoanewstatebasedonanearlyconstant turn-rate(NCT)transitionmodel,ordisappearswithprobability0.02.TheNCTobjectstatetransitionmodelcanbewritten as

˜

xt=F

(

^wt−1

)

^x˜t−1+Ut−1,wt=wt−1+

^uw (32) where

F

(

^w

)

⁼

⎡

⎢ ⎣

1 sin

(

^w

)

^{w−1 0} 0 cos

(

^w

)

⁰ 0

0

w⁻¹

(

^{1− cos}

(

^w

) )

sin

(

^w

)

^{w−1 10}

w⁻¹

(

^cos

(

^w

)

^{− 1}

)

−sin

(

^w

)

sin

(

^w

)

^w−1 cos

(

^w

)

^w−1

⎤

⎥ ⎦

^,Ut−1=

⎡

⎢ ⎣

²

2 0

⁰

0 0

²

2

⎤

⎥ ⎦



ux

uy