Área 4: Capacidades de gestión Gobernabilidad Democrática

Similar to PLS, the two-block CCA finds linear combinations of rows ofX

1

andX

2

maximizing

sequentially from the optimization problem:

{~a

(i)₁

, ~a

(i)₂

}=

arg max

~a1∈Rd1,~a2∈Rd2

h~a

|₁

X

1

, ~a

|2

X

2

i

subject to the constraints:

k~a

|₁

X

1

k= 1,k~a

|2

X

2

k= 1

h~a

|₁

X

1

,(~a

(j) 1

)

|

X

1

i= 0,h~a

|2

X

2

,(~a

(j) 2

)

|

X

2

i= 0, j

= 1,· · ·

, i−1.

(4.10)

This form makes the relationship between (4.5) and (4.10) clear (differing only constraining the

loading loadingsk~a

_k

k= 1 versus the scoresk~a

|_k

X

k

k= 1) and is equivalent to the usual formulation

of optimizing the correlation.

There is an important relationship between CCA and PAA (Hotelling, 1936; Bj¨orck and Golub,

1973), i.e., if

ρ

i

=h(~a

(i)₁

)

|

X

1

,(~a

₂(i)

)

|

X

2

i

is the

ith canonical correlation,

ρ

i

= cos(θ

i

), where

θ

i

is

theith principal angle between row spaces ofX

1

and

X

2

. The principal vector pairs{~x

1,i

, ~x

2,i

}=

{X

|₁

~a

(i)₁

,X

₂|

~a

(i)₂

}are often obtained through SVD ofV

_X|

1

V

X2

. In particular, let~u

X1,i

, ~u

X2,i

be the

ith left and right singular vectors ofV

|_X

1

V

X2

. Then, theith pair of principal vectors are

~

x

1,i

=V

X1

~u

X1,i

,

~x

2,i

=V

X2

~u

X2,i

.

An issue with CCA of high-dimensional data is related to the fact that CCA is interested in

the canonical vectors~a

i

rather than the principal vectors

~x

i

. In particular, when

d

1

> n, d

2

> n,

the values of~a

i

in (4.10) are not identifiable due to the singularity of

X

1

X

|1

and

X

2

X

|2

. Several

approaches have been taken to solve this problem. One approach is to use the Moore-Penrose pseudo

inverse in place of the inverse of

X

1

X

|₁

and

X

2

X

|₂

. A second approach is to add a ridge penalty

on

X

1

X

|₁

and

X

2

X

|₂

(Vinod, 1976). A third approach called

penalized CCA

is to add penalty

functions on

{~a

(i)₁

, ~a

(i)₂

}, such as an

`

1

penalty (Parkhomenko et al., 2007; Lˆe Cao et al., 2009),

an elastic net (Waaijenborg et al., 2008) or a fused lasso (Witten et al., 2009). Another approach

called

diagonal penalized CCA

is to replace

X

1

X

|1

and

X

2

X

2|

by diag(X

1

X

|1

) and diag(X

2

X

|2

)

(Parkhomenko et al., 2009; Witten et al., 2009).

Another important issue with CCA, which is directly related to AJIVE, is that when

d

1

>

n, d

2

> n, CCA is generally driven by noise. Lee (2007); Samarov (2009); Lee (2016) study the

inconsistency phenomenon in this case. One solution to this issue, used by AJIVE and COBE, is to

replace

X

k

by its low rank approximation ˜A

k

,k= 1,2. Theith principal vectors are~p

i

= ˜V

1

~u

1,i

,

~

q

i

= ˜V

2

~u

2,i

, where~u

j,i

is theith singular vector of ˜U

i

of the SVD of ˜V

|₁

V˜

2

respectively.

As described in Section 4.3, AJIVE uses an equivalent principal angle calculation based on

SVD of

M= [ ˜V

1

,V˜

2

]

|

=U

M

Σ

M

V

_M|

(Miao and Ben-Israel, 1992). AJIVE uses the transpose of

the

ith right singular vector,

~v

M,i

, as the estimated

ith basis vector of the joint space, provided

that the

ith principal angle is smaller than the threshold derived in Section 3.3.3.1. Moreover, if

the

ith principal angle has a value distinct from other principal angles, then the

ith left singular

vector of

Mcan be written as

~u

M,i

= [~u

|1,i

, ~u

| 2,i

]

|

/

√

2. Consequently

~

v

M,i

=

1

σ

M,i

M

|

~u

M,i

=

1

√

2σ

M,i

( ˜V

1

~u

1,i

+ ˜V

2

~u

2,i

) =

1

√

2σ

M,i

(~p

i

+~q

i

).

This shows that the AJIVE direction~v

M,i

is the scaled sum (essentially the average) of theith pair

of principal vectors.

CCA applied to the low rank approximations ˜A

k

and AJIVE are therefore closely related.

However, AJIVE provides one joint vector per two distinct principal vectors that by the virtue of

being an average should be a better estimate of the joint space than either of the principal vectors.

More importantly, AJIVE uses a theoretically sound threshold of the principal angles that allows

us to segment individual and joint variation.

multiset CCA

The AJIVE formulation allows for a natural extension to multi-block situations.

Several approaches of Multiset Canonical Correlation Analysis (mCCA) have been developed as

extensions of CCA (Horst, 1961; Kettenring, 1971; Nielsen, 2002). There is no general consensus

on which of these extensions is preferable. We point out that AJIVE is closely related to one of

the mCCA approaches discussed in (Nielsen, 2002).

This version of mCCA is defined using the optimization problem for the

ith set of canon-

ical vectors

{~a

(i)₁

,· · ·, ~a

(i)_K

}

and corresponding principal vectors (also called canonical variables)

{X

|₁

~a

(i)₁

,· · ·,X

|_K

~a

(i)_K

}:

{~a

(i)₁

,· · ·

, ~a

(i)_K

}= arg max

~a1,···,~aK

X

1≤k,l≤K

h~a

|_k

X

k

, ~a

|_l

X

l

i

subject to the constraints:

K

X

k=1

k~a

|_k

X

k

k

22

= 1,

h~a

|_k

X

k

,(~a

(j)_k

)

|

X

k

i= 0,

k= 1,· · ·, K,

j

= 1,· · ·

, i−1.

(4.11)

Notice that the constraint in (4.11) is different than the perhaps more naturalk~a

|_k

X

k

k

22

= 1 for all

k.

If the

ith singular value corresponding to the AJIVE direction~v

M,i

has a value distinct from

other singular values in the AJIVE SVD, then calculations similar to the two block case show that

theith basis vector of the joint space from AJIVE

~v

M,i

=

1

σ

M,i K

X

k=1

X

|_k

~a

(i)_k

is again a scaled sum (essentially average) of the corresponding canonical variables.

In document La gestión municipal, propuesta de mejora para la Municipalidad Provincial de Bolívar, 2017 (página 99-104)