Analysis and forecasting of time series by averaged scalar products of flow vectors

Complex Systems 8 (1994) 25-39

Analysis and Forecasting of Time Series by Averaged Scalar Products of Flow Vectors

C. SantaCruz R. Huerta

Instituto deIngenierfadel Conocimiento, UniversidadAut6nom ade Madrid,

Canto Blanco,Mod.C-XVI,PA,28049 Madrid,Spain

J.R. Dorronsoro Vicent e Lopez

Departamento deIngenierfaIn formatica, UniversidadAut6nomadeMadrid, CantoBlanco,28049 Madrid,Spain

Abstract. The relationship between the quality of state space re- const ructionand theaccuracy in timeseries forecastingis analyzed.

The averaged scalarproductof thedynamicalsyst emflowvectors has been usedto give adegreeof determinism tothe selectedstatespace reconstruct ion.Thisvalue helps distinguish between thoseregionsof thestate space where predictions will be accurate and those where theyare not . Atimeseries measur ed in an industri alenvironment wherenoise ispresentisused as anexample. Itis shownthat predic- tionmethods used to estimate futu re values play alessimportant role thana goodreconstruction ofthestate space itself.

1. Int ro d uct io n

Much work hasrecently beendonein nonlinear time seriesprediction [1-4]. However, most ofthis effort hasbeen focusedontime series originatingfrom dyn amicalsystems wher ethere is a well-defined, alt ho ugh complexandofte n analytic, underlying model. For such syst ems, a great number of different techniques have been developed to tackle the prediction problem. These techniqu esinclude state spacereconstructi on ofchaoticsyste ms [5-7]as well as various methods to approximate future values from eitherlocal [2,8]or globalmodels [1,9].

St a t e space reconstructi on is aimed at obtaininga trajectory inthestate space leadingto adeterministicreconstructi on ofthetimeseries. Thisisusu- allyaccom plished usin gTakens'theorem [10], whichstatesthat ifthe under- lyin g dyn amicalsyst em has dimension d,thereconst ru ct ion, usu all y called

26 C.SantaCruz,R. Huerta,J.R. Dorronsoro, and VicenteLopez

an embe dding, can be carr ied out usingdelay vectors in an m-dimensiona l space (m 2': 2d

+

1). Unlike the behav ior ofirregular time series, the state space usually demonst rates simplicity and regulari ty. Vector fields of the state space are approximated to est ima te future values . Thequality of the predictions strongly depend sonthequa lityofthestate space reconstructi on.

Furthermore, anylackofaccuracy or defect inthestate space reconstruction has consequences for the predicti on of timeseries. Therefore, a measur e of thequalityofa statespace reconst ru ction givesanest ima te ofthegoodness of the forecasting model. The average dscalarproduct P ofthe dyn ami cal sys- tem flow vecto rs [5] hasrecentlybeenused to determine optimalstate space reconstruction by maximi zing the P-valu e as a function of both the state space dimension and the time delay. TheP-valu e can beused to qualify the

"degree of det erminism" ofstate space reconstruction; tha t is, it provides a measure of how para llel the local flow vectors are throughout the state space. It has been shown [5]that for time seriesoriginating from ordina ry differenti al equa tions the P-value can be raised close to unity. This value indicates that the neighboringflow vectors in the reconstructed state space are para llel to each other. On the oth er hand, neighb oringflows for time seriescontamina te dwithnoise or originati ngfrom ill-defined systemswill be far from par allel. Thislead s to lower P-values. Thismeans that the syste ms are lessdetermini sti c tha n theform er,sincethe flowofpoint s alongthestate space are not sowell-defined. Hence,the predict ion will not be as accurate . Thiswork highlight s theexistingrelati onship between thecha ra cte ristics ofstate space reconstructi on of time seriesand the qualityof itspredictions in light of result sderived from the P-valu e. Some ofthe conclusions drawn from state space ana lysisapplyto both,substant iat ingthe result sobt ain ed inthe prediction and devisingnewpredicti on algorit hms .

As anexample,wewill use thestatespace reconstruction and time series prediction of a variable measur ed in an industrial enviro nment. This time seriescorrespo nds to an import ant temp eraturemeasur edinapetrochemical plan t. Forecastingisoneof the techniquesthat hasbeenint egrat ed intothe HINT project (ESP RIT) for intelligent control. Theseriesis sampledevery fiveminut es andthe timehistor yover three month s (25,000dat apoint s) is used . Figure 1 illustrat es20 hoursofthe actua l time seriesand thefilt ered signa l obtained afte r removing the high frequences using a low-pass filter.

Thefiltered signa lwill beused throughou t this paper.

Since the time series is measur ed in an industri al setting where a grea t number of uncont rolledfactor s are nodoubtpresent ,theunderlyingmodelis not well-defined. This lack ofaccuracy inthe definit ion ofthe system lead s tovarious problemsin both statespace reconstructi on and the qua lity ofthe predictionthat wouldotherwise not appear insit uationswhere a well-defined modelexisted.

The arti cle is structure d as follows. Section 2 describes problems that arisewhen performing statespace reconstructionsof realtime series. Section 3 ana lysesthe forecastingof timeseries usingresults obtained inthe previous section,and finally, section 4 summa rizestheconclusionsof this pap er.

Analysis and ForecastingofTimeSeries

- 10 , - - - - --.--- - --,---,---,

- 11

::

I I

----

^U 11^::\^:

a<::» :

12 i\

QJ -

H::J

-+-'CO HQJ

o, - 13

S I

QJ \,

E-<

~,:1~

-14

,

T¥

27

- 15

o

5 10

Time(Hours)

15 20

Figure 1: Vari ationof the original time series (dashed line) versus a filt ered time series (solid line) as afunctionof time. Temp eratureis measured in Celsius and is sampledevery fiveminutes.

2. Stat e space reconstruction

Stat e spa ce reconstructi onisnecessary before applyin gforecastingmethods.

This ana lysis provides afram eworkinwhichtomake state space reconstruc- tions,whichena bles us to approximate the flow vect ors .

St at e spa ce reconstruction starts wit h aunivari at e timeseries x;such as tempera t urevalues . This series is tra nsformed into a vector ^Xt. A map tha t transforms ^Xt int o x, must be chosen in such a way that points and flows maintain theirindividu alityinthe reconstruction:thatis,adet erministic and smo oth reconst ru ction is desired. The most accessible and robust method to perform state space reconstruction is the delay-coord inatemap [6]given by x; = (X t,Xt-r,...,Xt-(m-l)r) where ^T is the time delay and m is the state space dimension. Hence, the map is par am et erized by two var iables,

T and m tha t are chosensothat adet erministi creconstruction is obtained . Takens [10]showed that ifdis the dimension ofthe und erlying dyn ami cal syste m,an embe dding ofthe original time series^Xt can be obtain ed ina m- dimension al spacewherem ~ 2d+1andT is arbit ra ry. However , the presence ofnoisein any experimental measur e makesit difficult to realize the result of thetheorem. In practic e,we arefacedwit h an optimizationproblem that pursues the ideal{T,m}-par am et ers to obt ain the best system reconstruction.

The state space point s are rot at ed using theprin cipal component axis and

28 C.Santa Cruz,R.Huerta,J. R.Dorronsoro,and VicenteLopez each coordinate is scaled to the standa rd deviation ofthe new axis [5, 11].

The new statespacepoint s are called ^Zt.

The usu al method of determining the dimension of the st at e space re- construction is bymeansofthe corre lationdimension D2 . Thisdimension is a lower limit for the boxcounting dimension Do. Accordi ng to Man e

[12 ],

m > 2Do

+

1 is agoodestimate,but Saueret al. [7]suggest that m > Dois typicallysufficient (see Ot t [13J for a review). We calculate the correlation dimension in rot at edandscaled coordinatesz, as follows:

(1)

whereC(E) is thecorrelationintegral

(2)

and ()(.) is the Heaviside function, N is the numb er of points in the time series, and

I I . I I

istheEuclidean norm.

Experiment ally,D2is obtain edby studying the convergenceoflog2C(E) versus log,Eas afunctio nofm. InFigure2,log,C(E) is shownasafunction of both log2E and m. The plateau observed in the figure is due to the limited number of point s withinasphere ofsma ll radiusforlargedimensions.

When the e-value is increased, the number of point s within the sphere also increases and a more accurate value ofC(E) is obtained. In Figure 3, the slopes ofvarious log,C(E)versuslog,Ecurves are plot tedas afunctionof m and log2E. We see from this figure that there is no clear convergence to a constant slope. Theonlything we can be sure ofis that D2

>

8, that is,we needatleast eight dimensionsfor thestate spac ereconst ruction (m> 8).

In conclusion, this ana lysis does not yield solid evidence with which to model the system. It should be noted that the correlation dimension D₂ is an indirect measure of the embeddingdimension and does not take into account thedynamics ofthetime series,but only thepoints. Moreover,this timeseriescorr esponds toan ill-definedsyst em. Theseproblemsaccount for theunreliableestimat eobtained forthe underlyingdyn amicalsyste mofthis series.

On theot her hand,a deterministicreconstruct ion canbeobtained using the averagedscalar pro duct P.Thisvalue gives ameasure ofthenumberof self-crossings and tells how smooth the reconstructi on is. A value closeto unity implies a highly det erministi c and smooth reconst ruct ion, whereas a valueclosetozeroimpliesacompletelyrandomreconst ructi on. TheP-value is calculated by averaging the scalar product between the flow vector of a trajectorypoint andthe normalized flowvectors inside a ballofradiusEfor all points ofthe tra ject ory. The P-value is calculat ed as follows (see [5]for a moredet aileddescription).

Analysis and Forecastingof TimeSeries 29

-5 ,...

'-'"w

0eo'" - 10

...J0

-15

- 8

Figure 2: Three-dimensional plot of the variat ion of log2 C(E) as a functi on ofbothlog2Eand the state space dimensionm.

Define O(z;) as theset of neighboring points belonging to a closed ball ofdimensionm and radius E, tha t is,

O(Z;)

=

{Zj :

li z ; -

Zjll::;E,j

=

1,...,N,j

i-

i} ⁽³⁾

where JL; is the numberof points in O(z;). The norm alized flow vectors at each point arecalculated as

(4)

and the meanflow vectorvalueV;insidethe set O(z;)is given by

(5)

Finally, the scalar product is averaged along the trajectory using the expression

1 N

P = N L:f(z;) .Vi.

;=1

(6)

30 C.Santa Cruz,R.Huerta,J.R. Dorronsoro,and VicenteLopez

10 8

6

2

o

o

Figure3:Three-dimensionalplot ofthevariation oftheslopesof the variouscurvesof Figure2 as a functionof both log2Eand thestate space dimension m.

Think of the above calculations as a trip along the trajectorywit hin a tube of radius E, averaging the scalar product wit h alltheflows insidethis tube .

Since this time series has some isolat ed point sforeit her smallradiusEor high dimension m, we arbitrarilyfix adefault value off(Zi).Vi

=

0.2when no average is performed due to the lackof points wit hin the sphere. This servestodiscriminat ebetween the spurious resul t sdue toisolated points and thecorrect P-values. IfP convergesto 0.2as m increases,wemust increase

Einsuch a way that the numberof point sdoesnot vanishfor large m. Figure 4 illustrates the variation ofP for the original time series as a functionofbot h m and T. AvalueofE= 0.2isused. (Recallthata P-value close to unitymean s thatthesyste m ishighlydet erministic,whileaP-value closeto zero indicates arandom syste m.) A maximumP-value(P

=

0.55)is obt ain ed for m

=

4 andT

=

1. From this resul t , we concludethat the time series is not too det er ministi c. This P-value cont rasts with those obtained for syste ms originating from ordinarydifferential equations where P-values closetounityhavebeen obtaine d [5]. The reasonfor thisis twofold. Onthe onehand , there may be noise(high frequ encies) comingfromthesensor used tomeasurethetime series. Ontheother hand , there are manyuncontrolled factors that make the system ill-defined andgiverise to random sequences in the timeseries.

Analysis and ForecastingofTimeSeries 31

p 0.6

0.5 0.4 0.3 0.2

0.1

T

14

89 67 5 4 m 3 2 1

Figure 4: Three-dimensional plot of the variation ofthe averaged scalarproductsPof the originaltime series as afunctionofboththe statespacedimension m andtime delayT.

If we want to removethe noisedue to the presence of high frequ encies a low-pass filte r may be used. The use ofsuch a filter , however, has two disadvantages. First , the filtered signalobviously corres ponds to a different system. Secondly, thesefilters introduce adelay in the signal[14]inver ely prop ortionalto thepassband width that may beimp ortant in realtime ap- plications.Despit ethesedisad vantages, alow-pass filterwas used to remove high frequencies.

In Figur e 5,wedisplay the P-valuefor thefilteredtimeseriesas afunction ofT for different m-values . Avalueofe

=

0.2wasused inthe calculationof P. Themaximum value isobt ained for m

=

4 andT

=

1or3. Although the maximumP-valueisnearly the samefor both the originaland filt ered time series, the average P-value is much larger in the filtered series than in the originalone.

Acareful inspection of the reconstruction gives better insight int o what ishappeninginthistimeseries. Figure6showsaplot ofthesecond coordi- nate as afunctionofthe thirdfora 4-dimensionalstate space reconstruction.

The coordinate order is the one given by the principalcompo nent trans for- mation. From thisfigurewe conclude that therearetwowell-definedregions.

Thefirst oneis an inner region corresponding to a sphere ofradiusr :::::1.0 centered at the origin that is characterized by a grea t number of crossing tra jectories. This region corresponds to the mean value of the time series and theexcessive number ofcrossing tra ject ories might bedue to thepres-

32 C.SantaCruz,R.Huerta,J.R.Dorronsoro,and VicenteLopez 0.6

rn=4

p 0.5

0.4

0.3

m=3

m=5

< ,

-- _____rn=2

... ... ... ...

<,

0.2

1 2 3 4 5 6 7 8

T

Figure 5:Variationofthe P-valueofthefiltered signalas afunction of

T fordifferentstate space dimensions,rangingfromm = 2tom = 5.

ence ofahigher-dimensionalmovement or to randomfluctuations aroundthe mean, that is,noise. The second region, the outer one, is charac terized by a low level of crossing trajector ies. This observation makes clear the need to perform twodifferent estima tes ofP,one inside and another out side the sphere. In Figure7,one cansee that intheouterregion theP-valuereaches its maximum value for m

=

3while in the inner region a value ofm

=

6is obtained . Unliketheinner region, traject oriesintheouterregion arehighly parallel leading to a more det erministic reconstructi on. This implies that there is a moredeterministic syste m in the outer region ofthe state space, suggest ing the possibility that the system could be modeled with different dimensionsdependingon theregionwhere the point s are located.

3. Forecasting

In this section,we ana lyzetheinherent prediction limitations ofthisparticu- lar series inlight oftheresults obt ainedfrom the statespa ce reconstruction. The qual ity ofthe prediction is expressedin terms of thefollowing value:

E

=

Lt(Xt- £t)2 (7)

Lt(Xt- XtF

Thisratio is the norm alizedmean-squar ederrorwhosevalue isunitywhen each ofthepredict edvaluescorres pond to the avera ge. A ratioaboveunity

Analysis and ForecastingofTimeSeries 33 8

6 4 2

- 2 - 4

- 6 -8

- 8 -6 -4 2 4 6

Figure6: Variation of the second coordinate ofthe state space re- construction Z2 as a funct ionof the third coordinat eZ3 fora Four- dimensionalstate space reconstru ctionobtainedfromthe filteredtime series.

correspo nds to a predicti on that is worse tha n the average; a rat io below unityis an improvement overthe average.

Foragiven pointinthetimeseriesXtwe wishtopredict thenext npoints into thefutureXt+k,k

=

1,... ,n, wheren is the predicti on horizon . These est ima tes are determined using the previous points in the time series Xi,

i

=

1,....t,which arethen used tobuildthe statespace reconstruction. We use both the predicted values ^{Xt+ k} and the actual values ^Xt+k to det ermine theE-valueandthus to measuretheperformanceof different models.

We see that thecharact eristicsof thestatespace derivedusingtheP-value (6),havetheircounterpartswhenexamining the prediction behavior. These results highlight the relationship between the P-value and the prediction err orE.

Firstofall,we det ermine theoptimumstate space dimension usingthe prediction erro r. In additio nto the metho ds used intheprevious sectionto determine thestatespacedimension,another possibilityis todeterminethe performanceofthe reconstruction whenestimating thetimeseriespredicti on error[1]. In order to dothis,we choose thesimplest prediction method,that is, alocal approac h usinglinearpredictors [2,8].

34 C.Santa Cruz,R.Huerta,J.R.Dorronsoro, and VicenteLopez 0.7

p 0.6

0.5

0.4

0.3

0.2

0.1

0.0 1

,j f

/

;

;

2 ,,,, ,, ,

3 4 5 6 7 8 9

m

Figure7: Variation ofthe P-value as a function of the sta te space dimension for boththosepointsthat lieoutside a sphereof unitradius (solid line)andthose that areinside (dashed line).

The procedure used to det ermine the predictionis describ ed as follows.

For each point ofthe time series, locate 50 nearb y m-dimensionalpoint s in the state space . The prediction is obtained by fitting a linear predictor to these 50 point s toget her wit h the points where they end up afte r one ste p into the future. The predict ed value is nowtaken as the new point to be predicte d and the procedure is repeated until the prediction horizon n is reached. The linear fit is obt ain ed using singular value decompositi on to avoid ill-condi tioned sit ua t ions.

Figure 8 shows the prediction err or, calculated for a prediction horizon ofn = 1, as a functio n ofthe state space dimension m. It is seen that the erro r decreases as m increases,to a minimum value at m

=

4. Tha t m

=

4 is the optimum dimension for makin g predicti ons is in agreement wit h the result sobt ain ed inthe previous section using average dscalar product s. The predicti on erro r E decreases as the state space dimension is increased up to the point where self-inte rsect ions are avoided . For this dimension , the best local fit is obtained. However, when increasing the m-value beyond the optimum,eventhoughself-crossing tra ject ories are thus avoided,points in the resulting high-dim ension al space are too dist an t from each other to ensure a good local approxima ti on .

AnalysisandForecastingofTimeSeries

- 3.0 ..-- . - - - -- , - -- , - ---r---,-- r ----, 35

- 3. 1 -3.2

-3.3

w

₀

-3.4 eo~

~0

- 3.5 -3.6 -3.7

2 3 4 5 6 7 8 ⁹

ill

Figure8:Variation of thepredictionerrorE obtained foraprediction horizon ofn= 1as afunctionofthestate spacedimensionm.

We now examinetheconsequencesthis reconstruction has on both short andlong-t ermpredictions.

We have used different prediction algorithms to estima te future values ofthe timeseries,includi ngbothglobalmodels (neuralnetworks) and local methods that use both linear and nonlinear approximations. In all cases, theprediction errorswerenot significantlydifferent. Theseresults highlight the fact tha t the predicti on algorithm was not as important as the stat e space reconstruct ion. Consequently, the method previously described was singledout. Althoughtheoptimalstatespacedimensionwasdetermined for a prediction horizon ofn

=

1,we assumethat thisdimension isnot far from theoptimum forlarger predictionhorizons.

Figure9illust rat es short-termpredictions(n= 10) forseveraltimeseries compared with the actua lvalues. Theconclusions drawn from this figureare twofold. On the one hand, predicted values compar e quite favorably wit h the actualvaluesforthosepointsofthe timeseries that corres pondtolarge peaks. On theother hand, predictions for those values close to the avera ge

T =

- 12.8°C are not so good . The reason for this is clear if one takes into account the state space reconstruction. It was shown that the core of thestate space correspo ndstohigher-dimensionalmovement whiletheouter region is more det erministic (Figure 6). Therefore, it is difficult to obtain a det erminist ic reconstruction for the inn er region ofthe state space and,

36 C.Santa Cruz,R. Huerta,J.R. Dorronsoro,and VicenteLopez - 10

- 1 1 !

1\ II

n

-r--;

1\

~

^1\

i \

u ,.

)\

a"-"

12

(l) -

;::JH -+-'

(1j H

(l)

P< ^- 13

S

(l)

Eo-<

- 14

-15

o

^{5 10}

Time (Hours)

15 20

Figure 9: Vari ati on of theorigina ltime series (das hed line) and the filtered timeseries (solid line) along with thepredict ed valu esfor a predicti onhorizon ofn= 10 (circles).

as a consequence,predictions are inaccur at e. On theother hand,theout er regionofthe state space,whichcorres po ndsto the various peaksthat appear in the time series,can be det ermini sti cally reconstruct ed and are therefore quit ereliable. This result can behighlight ed ifwedo not makeprediction s forthosevaluestha t are located in the core ofthe state space. Taking into account the overalltimeseries wefind tha t E

=

0.092. On theoth er hand , if only those values of the time series whose state space points lie outside a sphere of radiu sr = 1.0are predicted,the error decreases to a value of E = 0.074. This mean s that the mean-squar e error decreases by a factor of20%ifthe inner region of thestate space (representing 30%of the total numberofpoint s)isignor ed.

We now tackle the probl em oflong-t erm predicti ons. Figure 10shows the predicti onsfor^ti= 100steps int o thefut ure compa red wit h theoriginal and filtered time series. In this figur e, two complete 100-st ep predicti ons are shown . The prediction s start at t :::::; 3h and at t :::::; llh,resp ectively, for a duration of100points each . ^It is seen that both peaks are accurately predict ed.These peaks correspondto closedorbitsin the state space recon- struction (Figure 6). The remaining peaks of the figur e are not accurately estimated duetothefact thatwhenthe predicti onreaches theaverage value

T

= - 12.8°C afterapproxima te ly 15 steps into thefuture(which corresponds

Analysisand Forecastingof Time Series 37 -10

15 20 10

Time(Hours) 5

-15

o

l

H

- 11

1\

fl

---._U ^1\:i

0 j'

'---'

- 12

Q)

H;::J +-'cd H

Q)

c, - 1.3

S

Q)

Eo-<

-14

Figure 10: Variation of the original time series (dashed line), the filtered timeseries (solidline), and the predicted valueswith a pre- dictionhorizon ofn= 100(circles).

toente ring the inn er regionof the state space) it can not escape and begins to oscillate. Thus, the predicti on horizon for thissystem is limited by the time required to reachthe innerregionofthestatespace . Once there,due to the grea t number ofcrossingtrajectories,there is noway out of the region, and hencethe remaining peaks arepoorlyest imated .

Driven by the discovery that two well-defined regions wit h different di- mensions aredisti ngu ished inthestate space,webuilt a modelthat takes this int o account. In this model, eit her a three or a six-dimensiona lstate space is useddependi ngon whet herthestate space point lies outsi deorinside the core region. TheE-va lue obt ained forthis predicti onmodelis only5% better than the former. Thisslight difference is not significant and suggests tha t the innerregion is mainl y domin at ed by noise inst ead ofbeing a movement embedded in a higher dimension. The mode lmight be more successful for thosesystems characterized by alow presence ofnoise.

4. Discussion

The result s obtainedsuggesttha t theP-valu e can beint erpr eted as ameasur e of the degree ofdet erminism of thestate space reconstructio n. Furt her more, the P-valu e gives a measur e of the qualit y ofthe reconstructi on for differ-

38

G .

SantaCruz,R. Huerta,J.R. Dorronsoro,and Vicente Lopez

ent regions ofthe state space that distinguishes between non-det erministic regionsdominat ed bynoiseand highlydet erministicregions.

Ithasbeen shown that the result sobtainedinthestatespace reconstruc- tion have their counterpart in the prediction err or method indicati ng the similarities between the localavera gedscalar product value and the predic- tionerro r. Thisresult suggest sthepossibili tyofusing this value to estimat e alocalpredicti on err or.

The procedure described in this paper out lines a method for modeling high-d imension al syste ms wheredue to the lack of points abad estimateof Dz is obtained. However, a degree of det erminism and hence a degree of smoot hness of the reconstructionmaybeobt ain edusing theavera gedscalar products of flow vectors.

In this workand forthisparticulartime series,different predictionmodel includingglobalmodelsusingneur alnetworks,local modelsusingboth non- linear approximations suchasneuralnetworks andlinear methodshavebeen tested, leading in allcases to almost the same resul t. However , localmod- els usinglinearapproximati ons outperform theothersin term sof execution time,along wit hthe possibility to useit in adaptivereal timesyst ems. This result emp hasizesthe fact thatagood reconstructi on iscritical,whereasthe metho d used to modelthe localflow vectors is notso important.

5. Acknowledgments

Grat eful acknowledgment is due to the ESPRIT project HI T E6447 for support ofthis work. One of us (R. H.) wan ts to thank to the M.E.C.for financialsupport.

Reference s

[1] M.Casdagli, "Nonlinear Predictionof ChaoticTimeSeries," Physica D,35 (1989)335-356.

[2] J.D. Farmer andJ. J.Sidorowich, "PredictingChaoticTimeSeries,"Physical ReviewLett ers,59 (1987)845-848.

[3] J.D.Farmer andJ. J.Sidorowich, "Exploiting Chaos to PredicttheFutur e and Reduce Noise," in Evolution, Learning and Cognition,edited by ^Y. C.

Lee(Singapore:World Scientific,1988).

[4] J.P.Crutchfieldand B. S. McNamara, "Equationof Motion fromDataSe- ries," ComplexSystems, 1 (1987) 417-452.

[5]R.Huerta,C. Sant aCruz,J.R.Dorronsoro,andV. Lopez, "State SpaceRe- constru ctionUsingAveragedScalarProductsof theFlowVectors," Physical ReviewE,49 (1994) 1962-1967.

[6]^{j .} H. Packard ,J. P. Crutchfield,J.D. Farmer, andR. J. Shaw, "Geometry from TimeSeries," PhysicalReviewLett ers,45 (1980) 712-716.

AnalysisandForecastingofTimeSeries 39

[7] T. Sau er,J.A.Yorke, and M.Casdagli,"Embedeology," Journ alof Statist ical Physics,65 (1991)579-616.

[8] T. Sau er , "T ime SeriesPredicti onbyUsingDelay Coord inates Embedding,"

in Tim e Seri es Prediction: Forecasting the Future and Understanding the Past,edited byA. S. Weigend andN. A.Gershenfeld (Rea ding,MA: Addison Wesley,1993).

[9] V.Lopez,R.Huerta andJ.R. Dorronsoro ,"Recurrent and FeedforwardPoly- nomial Modeling of Coupled Time Series," Neural Computation, 5 (1993) 795-811.

[10] F.Takens, "Dete cti ng Stran geAttractorsin FluidTurbulence,"in Dynamical SystemsandTurbulen ce,edited byD. Randand L.S. Young(Berlin: Springer, 1981).

[11] D.S. Broomhea d and G. P. King, "Extractin g Qua litative Dyn amicsfrom Exp erimental Dat a," PhysicaD,20 (1986) 217-236.

[12] R. Mane, "Onthe Dimension ofthe CompactInvariantSet sofCert ainNon- linear Maps," inDynamical Systems and Turbulence,edited byD. Rand and L.S.Young (NewYork: Springer,1980).

[13] E. Ott , "Chaos in DynamicalSyst ems" (Cambridge : Cambridge University Press,1993).

[14] A. V. Oppenheim andR. W.Schafer,"Discrete -t ime SignalProcessing,"Pren- ticeHallSignal ProcessingSeries (Englewood Cliffs,NJ :Prenti ceHall,1989).