1 2 7T( P ) 7T( P ) 2 3 With th is s e ttin g , r a n k ( K (p)) would be
Hannan and Deistler(1988) proved t h a t equal to th e de g re e of t h e polynomial detA (z ) which is ^ hp. Moreover,
p if th e r a n k of K p is f i n i t e th e n r a n k ( K (p) ) = r a 7T(P) TT(P) TT(P) 1 2 P 7T(P) rr(p) • • • 7T(P) 2 3 P + 1 7T(P) rr(p) • • • TT(P) P p + i 2 p- 1
It th e r e f o r e s u g g e sts t h a t we should c o n c e n tra te on K (p) and o u r
p
o b je c tiv e becom es to o b tain th e b e s t a p p ro x im a tio n of r (p)= ra n k (K (p)) f o r p = 1 ,2 ,...,m ax. Since o ur u ltim a te goal is to
p p
id e n tify a s u ita b le m odel f o r th e d a ta , th e ra n k should be s e le c te d in te r m s o f th e e x p la n a to ry pow er of th e fam ily of m odels w ith th e ra n k . How ever, by in c o rp o ra tin g a h ig h er ra n k , one u su ally can g e t a b e tt e r e x p la n a to ry pow er. T hus, th e ra n k se le c tio n should fa v o u r a s m a lle r ra n k th a n a la rg e one w hen th e ir fa m ilie s of m odels have s im ila r e x p la n a to ry pow er. F o rtu n a te ly , th is jo b can be f u lf ille d by fo llo w in g a rg u m e n ts fro m C rabbe and Young(1989) who c o n sid e red a p p lic a tio n o f S in g u la r Value D ecom position (SVD) and In te rn a lly B alanced R e a lisa tio n to model re d u c tio n . B asically , th e i r id ea is sim ple. R e p re se n ta tiv e m em bers fro m each fa m ily o f m odels of d if f e r e n t ra n k s a r e s e le c te d and com pared. The ra n k a t w hich th e c o rre sp o n d in g r e p r e s e n ta tiv e p ro d u ces th e lo w e st v alu e o f a c r ite r io n fu n c tio n is s e le c te d . At th is p o in t, we w ould like to p u t a re m a rk f o r th o se r e a d e r s who th in k s t h a t th e m odels id e n tifie d by m axim um likelihood e s tim a to rs would do th e job. T his, in f a c t , may n o t be th e c a se . F o r one th in g , m axim um likelihood e s tim a tio n re q u ir e s in itia l e s tim a te s w hich is w h a t we w ould like to fin d . A nother p roblem is t h a t th e m odels id e n tifie d by m axim um likelihood e s tim a to rs m ay n o t be n u m erica lly w e ll-b e h av e d . In ad d itio n , th ey may n o t p o ssess s im ila r p r o p e r tie s . T h e re fo re it may be d if f ic u lt to sin g le out th e e f f e c t due to in c re a s e in dim ension.
To fo rm u la te th e id ea fro m C rabbe and Young(1989), we need th e fo llo w in g d e fin itio n s:
We co n sid e r th e fo llo w in g s t a t e sp ace model: y ( t) = C x (t) + e ( t )
x ( t+ l) = A x(t) + B e(t) D efin itio n 5 .2 .2
For s t a t e sp a ce m odel (5.2.1), we d e fin e th e o b s e rv a b ility and th e c o n tro lla b ility a s 0 = (C* (CA)’ • • • • )’ and £ = (B AB A2B • • • • )
re sp e c tiv e ly . F o r convenience we w r ite , f o r n e IN,
o = ( c (c ay • • • ( c A n_1)’ r
n
R em ark:
1. S y stem (5.2.1) is o b se rv a b le and c o n tro lla b le if 0 and t? have
n n
f u ll ra n k n w h ere n = d im (x (t)). 2. K (p) =
p p p
D e fin itio n 5 .2 .3
The o b s e rv a b ility and c o n tro lla b ility g ram m ia n s of sy ste m (5.2.1) a r e d e fin e d a s O' 0 and re sp e c tiv e ly w h ere n = d im (x (t)).
n n n n
D e fin itio n 5 .2 .4
The sy ste m (5.2.1) is c a lled in te rn a lly b alan ced if and only if i ts o b s e rv a b ility and c o n tro lla b ility g ram m ia n s a r e equal.
Given th e dim ension n o f x (t) and th e tr u n c a te d Hankel m a tr ix H , we can id e n tify th e in te rn a lly b alan ced r e a lis a tio n in th e fo llo w in g way:
S te p 1
P e rfo rm a SVD on H to o b tain K = US V’ w h ere U and V a re
n n n
o rth o n o rm a l m a tric e s , i.e. U,U=V’V=I and S = d iag(s , . . . , s ) w here
n 1 nh h = d im (y (t)). D efine ( 5 . 2 . 5 ) H = B rr l H =(n C 1 • n ) andn n n 2 n 3 TT 3 n 4 • TT P + 1 • n p +2 n n n p + i np +2 • TT2 P S te p 2 W rite A = S ' ^ U ’K^VS"172, B = S"1/2U’H and C = H V S '1/2 n n n n B C n
Then w e can show t h a t th e o b s e rv a b ility and c o n tr o lla b ility g ra m m ia n s o f th e m odel d e fin e d above a r e both eq ual to S^. M oreover, we have th e fo llo w in g nice p r o p e r tie s w ith r e s p e c t to th e m odel d e fin e d in th e above way:
R e su lts (C rabbe and Young(1989))
1. A m odel id e n tifie d by s in g u la r values is in te rn a lly b alan ced.
2. All in te r n a lly b a la n c e d m odels a r e s ta b le if th e tim e s e r ie s i ts e lf is s ta b le .
3. The q u a n tity max{/i(d?),^i(&)> ach iev es i ts m inim um f o r in te rn a lly b a la n c e d m odel w h e re u(M) = s / s and s and s a r e th e
la r g e s t and s m a lle s t s in g u la r v alues of M resp e c tiv e ly .
* N ote t h a t th e h ig h er th e co n d itio n num ber ji( •) th e m ore ill-c o n d itio n th e p roblem is.
The above r e s u lts su g g e st t h a t all in te rn a lly b alan ced m odels a re n u m e ric a lly w ell-b e h av e d and g u a ra n te e s ta b ility . M oreover, if th e in te r n a lly b a lan ced m odels w e re id e n tifie d by s in g u la r valu es th e n th ey a r e g u a ra n te e d to be o b se rv a b le and c o n tro lla b le , i.e. of m inim al dim ension. C onsidering m odels of such q u a litie s can sin g le o ut th e e f f e c t due to d iff e re n c e in th e dim ension of s ta t e v e c to r.
F o r p = 1,2... m ax ., p th o rd e r a u to re g re s s io n g e n e ra te s a Hankel m a tr ix a p p ro x im a tio n H (p) w ith ra n k ^ hp. Suppose t h a t p e rfo rm in g SVD on K p id e n tify s , . . . , s a s s in g u la r v alues. L et
p 1 hp
1 < n ^ hp, d e fin e S = d iag (s , . . , s ) and S = d iag (s , . . , s )
1 I n 2 n+1 hp
w ith U = (U U ) and V = (V V ) th e c o rre sp o n d in g p a rtitio n . Then
1 2 1 2
we c a n c o n sid e r th e in te rn a lly b alan ced model id e n tifie d by s , . . . , s d e te rm in e d in th e fo llo w in g way: 1 n y ( t) = C^xit) + e ( t ) x ( t+ l) = A x (t) + B e (t) l l w h ere 1. n =: d i m ( x ( t ) ) 2 . A = S~1/2U’K (p) l i i p 3. B = S"1/2U’H (p) l 1 1 B 4. C = K (p)V S '1/2 l C 1 1 l l R em ark: H (p\ K (p), H (p) a r e d e fin e d in (5.2.5). p B C each o f th e = 1 ,2 ,...m a x ., By c o m p a rin g v alu es o f a c r ite r io n fu n c tio n s of
in te rn a lly b a la n c e d m odels f o r n = l ,2 ,...,h p and p
we c an s e le c t th e dim ension n w hich w ill give th e b e s t f i t to o u r d a ta . In th e l i t e r a tu r e , th e m o st commonly em ployed c r i t e r i a f o r o rd e r s e le c tio n in tim e s e r ie s a n a ly s is a r e AIC, BIC and HQ w hich have th e fo llo w in g g e n e ra l fo rm :
(5.2.6 ) CF(n) = -21 o g-lik elih oo d + C^d(n)
w h ere n is th e dim ension u n d e r c o n s id e ra tio n , Ct is a b a la n c in g c o n s ta n t (to be d iscu ssed ) and d(n) is an in c re a s in g fu n c tio n of
n. AIC, BIC and HQ c o rre s p o n d to = 2, = lo g T and C = 2 c lo g lo g T , w h e re c > 1, re s p e c tiv e ly . The r a tio n a le beh in d (5 .2 .6 ) is t h a t a lth o u g h -2 1 o g -lik e lih o o d can be re g a rd e d as a m easure o f goodness o f f i t , it s value decreases as th e d im e n sio n in cre a se s. Hence, an a d d itio n a l p e n a lty te r m should be added to p re v e n t th e s e le c tio n o f la r g e r th a n necessary o rd e r. One o f th e p ro m in e n t c r i t e r i a f o r choo sin g Ct is t h a t CF(n) w i l l e v e n tu a lly p ic k th e
c o r r e c t o rd e r w hen m ore and m o re in fo r m a tio n a re com ing. I f we can use th is as o u r p r in c ip a l c r it e r ia f o r choosing C^, we have th e f o llo w in g r e s u lt due to H annan(1984):
T h e o re m (5 .2 .7 )
Assum e t h a t th e re is a t r u e n and n m in im is e s (5 .2 .6 ) w h ile , as
o
T —» co, C ^ /T —> 0. Then th e fo llo w in g holds, w h e re a.s. s ta n d s f o r "a lm o s t s u re ly ".
( i) I f lim i n f C /(2 1 o g lo g T ) > 1 th e n n — > n a .s ..
t— X» t o
I f l im ^sup CT/(2 1 o g lo g T ) < 1 th e n n does n o t converge a.s. to n . o ( ii) I f l im i n f C = oo th e n n —> n in p r o b a b ility . T XX) t o K J I f l im sup C < oo th e n l i m l i m P(n > n ) = 1. t— X» t _ o <5->0 T->oo
We observe th e f o llo w in g f a c t s f o r AIC, BIC and HQ:
1. AIC is n o t a c o n s is te n t c r i t e r i a and w o u ld te n d to o v e re s tim a te th e tr u e o rd e r o f th e system .
2. BIC and HQ a re b o th s tr o n g ly c o n s is te n t. H o w ever, u sin g Q u in n (1 9 8 0 )’ s w o rd s , BIC w i l l te n d to u n d e re s tim a te th e tr u e o rd e r in s m a ll sam ples, r e la tiv e to th e HQ c r it e r ia , w h ic h is a ls o s tr o n g ly c o n s is te n t, b u t m in im a lly so.
I f we can ju d g e d if f e r e n t c r i t e r i a fr o m t h e ir c o n s is te n c y th e n th e ch o ice is r e a lly b e tw een BIC and HQ (assu m in g c o n fin e d to th e th re e c r it e r ia ) . B u t p r a c t ic a l e x p e rie n c e d e m o n s tra te d t h a t BIC w i l l p ic k an o rd e r w h ic h is e qual to o r less th a n th e o rd e r p ic k e d by HQ. In t h is sense, HQ te n d s to be m o re c o n s e rv a tiv e . We a re , th e r e fo r e , in c lin e d to HQ c r it e r ia . A n o th e r im p o r ta n t d e c is io n we