2.4.1
Duplicate invariance property
Garthwaite et al. (2012) showed that the cos-square transformation has a dupli- cate invariance property. Suppose we have two sets of vectors x1, . . . , xp and
x1, . . . , xp, x(p+1), . . . , x(p+k) where the first set is a subset of the second set. If
both sets are transformed separately then the transformed values of x1, . . . , xp in
the two sets are likely to differ. This is also true for the case where the original sets are transformed to orthogonal sets, the only exception is when the vectors
x(p+1), . . . , x(p+k) are orthogonal to x1, . . . , xp. However, suppose each of the vec-
tors x(p+1), . . . , x(p+k)is an exact duplicate of xp. Then the transformed orthogonal
vectors associated with x1, . . . , x(p−1) obtained under the cos-square transforma-
tion will be the same in the two transformed sets. That is, duplication of xp has
no effect on the transformed values of x1, . . . , x(p−1). Also the transformed values
of x1, . . . , x(p−1) are virtually the same if each x(p+1), . . . , x(p+k) is virtually a du-
plicate of xp, i.e., if xl = xp+ αζl for l = p + 1, . . . , p + k, where α ≈ 0 and ζl can
be any value. Garthwaite et al.(2012) called this property of the cos-square trans- formation the ‘duplicate invariance property’. For more details of the duplicate invariance property, see Garthwaite et al.(2012).
2.4.2
Rotation invariance property
When there is a strong collinearity between some of the X variables, then some columns of the transformed orthogonal data matrix are not close representation of the corresponding columns of X. Orthogonal rotation of collinear variables can remove collinearities but, with most transformations, rotation of some variables will typically affect the transformed values of all variables. However, as noted in the introduction, the cos-max and the corr-max transformations have a rotation invariance property. In this subsection we first show that rotation can reduce collinearity and then describe the rotation invariance property.
An orthogonal rotation of axes X1, X2 to axes X1∗, X ∗
2 is illustrated in Figures
2.1(a) and 2.1(b). In Figure 2.1(a), the positions of 10 points (x1, x2) are plotted
and new axes X1∗ and X2∗ are shown. The new axes are obtained by rotating the original axes X1 and X2 (by 45o in this case). Figure 2.1(b) shows the same 10
X1
X1∗ X2
X2∗
Figure 2.1(a) Points before rotation with X1, X2 as axes.
X1∗ X2∗
Figure 2.1(b) Points after rotation with X1∗, X2∗ as axes.
points, but drawn with X1∗ and X2∗ as the horizontal and vertical axes. It can be seen that rotation of axes changes the correlation between variables: Figure 2.1(a) shows that the points are highly correlated when expressed in terms of X1
and X2, while Figure 2.1(b) shows that the correlation is low when the points are
expressed in terms of X1∗ and X2∗. Consequently, orthogonal rotation can be used to remove or reduce collinearity between variables.
We only need to rotate those variables that are involved in collinearities. For example, suppose there is just one collinearity and it involves only the first d variables X1, . . . , Xd. Then axes are rotated using a rotation matrix, Γ say, that
has the following block-diagonal form:
Γ = Γd 0 0 Ip−d , (2.33)
where Γd is an orthogonal matrix of order d and Ip−d is a (p − d) order identity
matrix.
Rotation produces new variables that are linear combinations of the original variables. The rotation matrix should be chosen in such a way that the variables
that are created have meaningful interpretation. For example, if only the first two variables X1 and X2 are responsible for one collinearity then Γd can be set as:
Γd= 1 √ 2 − 1 √ 2 1 √ 2 1 √ 2 (2.34)
This rotation creates two meaningful variables, the first one is proportional to X1+ X2 and the second one is proportional to X2− X1.
In terms of the original variables, X1 and X2, the ten points in Figure 2.1 form
the data matrix: −0.48 −0.42 −0.24 −0.18 −0.12 0.18 0.18 0.24 0.36 0.48 −0.48 −0.41 −0.41 −0.07 0.07 0.14 0.21 0.14 0.41 0.41
Post-multiplying this data matrix by Γd in equation (2.34) gives the points in
terms of the new variables X1∗ and X2∗: −0.68 −0.59 −0.46 −0.18 −0.04 0.22 0.27 0.27 0.55 0.63 0.00 0.01 −0.12 0.08 0.13 −0.03 0.02 −0.07 0.04 −0.05
The sample correlation between X1 and X2 is 0.951, while the sample correla-
tion between X1∗ and X2∗ is 0. (The correlation between the sum and difference of two variables that have been standardised to have equal variances is always 0.)
To illustrate the rotation invariance property of the cos-max and the corr-max transformations, suppose the rotation matrix is of the form in equation (2.33). Let the data matrix after rotation be X∗, i.e., X∗ = XΓ. This rotation rotates only the first d columns of X, while the last p − d columns of X and X∗ are the same. If the cos-max transformation is applied separately to X and X∗ then the last p − d columns of Z and Z∗ are same, where Z and Z∗ are, respectively, the transformed matrices of X and X∗. That is, with the cos-max transformation, the rotation
of variables has no effect on the transformed values of the unrotated variables. This is also true for the corr-max transformations. Rotation of first d columns changes only the first d components of W , the remaining p−d components remain unchanged. Consequently the contribution of last p−d variables are unchanged by the rotation. Rotation can be performed before or after transformation and in both cases yields the same result. For a detailed description of the rotation invariance property of the corr-max transformation see Garthwaite and Koch(2016).