CONSIDERACIONES DE ORDEN GENERAL

I entered the data into an exploratory factor analysis with Mplus 6.12 using the maximum likelihood robust (MLR) estimator for complex survey data (i.e., adjusted for multilevel data). While principal components analysis (PCA) is a very common form of EFA, I instead employ common factor analysis. With PCA components are estimated to represent the variances of the observed variables in as economical a fashion as possible (i.e., fewer dimensions), with no assumption of latent variables underlying the observed variables (Floyd & Widaman, 1995). In contrast, with common factor analysis the factors are estimated to explain the covariances among the observed variables and the factors are seen as the causes of the observed variables. Since my research involves the use of latent variables (i.e., underlying dimensions), common factor analysis is a more sensible approach than PCA (Conway & Huffcutt, 2003). While this was a moot point

given that Mplus only utilizes common factor analysis, PCA was an available option in both Stata 13 and SPSS 21.

The default output includes the geomin oblique rotation for the factor loadings. Conway and Huffcutt (2003) noted that while the varimax orthogonal rotation is by far the most common kind of rotation for generating “simple structure” of the factor loadings (i.e., a more visually interpretable output of the factor loading matrix), they argue that this approach is often inappropriate. The varimax rotation attempts to produce some high loadings and some low loadings for each factor to achieve simple structure. However, Conway and Huffcutt (2003) argue that orthogonal rotations force unrealistic solutions that likely distort loadings away from simple structure and instead recommend oblique rotations. Mplus implements the geomin oblique rotation type (Browne, 2001) for much the same reasons (Muthén & Muthén, 2010).

The program was asked to fit between one and seven factors, inclusive. While five factors were hypothesized, the program was asked to fit up to seven factors for model comparison. Since Mplus implements the MLR estimator, fit indices are produced that can assist in the interpretation of the most appropriate number of factors. Although the Eigen values > 1 criteria has long been used in EFA as factor selection criteria (Floyd & Widaman, 1995), methodological research reviewing EFA has strongly discouraged this practice (Conway & Huffcutt, 2003; Fabrigar, Wegener, MacCallum, & Strahan, 1999; Lance, Butts, & Michels, 2006).

The first iteration of the factor analysis indicated that the six factor model exhibited the best model fit of the seven tested: χ2(185) = 248.505, p=.000, root mean square error of approximation (RMSEA) = .026, comparative fit index (CFI) = .989, Tucker-Lewis index (TLI) = .980. χ2, RMSEA, CFI, and TLI are indices used to indicate the quality of the model fit with the observed data. The χ2 statistic tests the null hypothesis that the model is correct in that it perfectly fits the population/observed data. Rejecting the null

hypothesis with a non-significant χ2 is desirable (Kline, 2005). While the χ2 test was significant, following conventional SEM recommendations for evaluating additional indicators of model fit, RMSEA values of below .08, and CFI, TLI values above .95

indicate acceptable model fit (Hu & Bentler, 1999). These supplemental indicators of model fit are often used given that the χ2 test is considered quite conservative and sensitive to sample size (Kline, 2005). In any case, χ2 results should be interpreted cognizant of the current limitations of model fit index guidelines in a multilevel context (Kostopoulos et al., 2013; Ryu & West, 2009). Table 21 shows the full geomin rotated loadings with loadings higher than .3 highlighted in bold.

The six factor solution showed that the OE, SA, and C items were mostly loading on separate factors as hypothesized. However, the LDS and ODC items were clearly cross loading (i.e., factor loadings were relatively high on more than one factor). Between the two sets of LDS and ODC items three factors were identified. To investigate this loading pattern I ran a separate EFA on the ten LDS and ODC items combined, requesting Mplus to generate up to four factors for these items. A three factor solution emerged which replicated the cross loading results from the six factor model. What was most glaring about this result was that the item ODC1 was loading very highly (.888) with LDS1 and LDS4 on one factor. I examined the wording of these items and drew the conclusion that these items seemed to collectively reflect an “innovation and improvement” aspect of learning and development support and opportunity driven change. ODC items 3,4, and 5 loaded well together on a single factor. The remaining items (LDS2,3,5, and ODC2) were cross loading among the factors greater than .3. As a result, I removed these cross loaded items and re-ran the EFA. This produced a two factor solution which fit the data well with the items clearly loading on two distinct factors.

Returning to the OE, SA, and C items from the six factor solution, I removed the items which were cross loading or had the lowest loadings in the group. This reduced the scale to three items per dimension for parsimony. The remaining items were OE1,2,5,

SA2,3,5, C1,2,4, and ODC3,4,5. Due to the high loading of item ODC1 with LDS1 and 4, it was renamed “LDS6” to form the LDS factor. An EFA was run on these items fitting up to a seven factor solution. A five factor solution best fit the data: χ2(40) = 53.715, p=.072, RMSEA = .021, CFI = .997, TLI = .992. Table 22 shows the full geomin rotated loadings with loadings higher than .3 highlighted in bold, as well as Cronbach’s alphas for each item grouping for internal consistency reliability. The factor loadings

Table 21 Geomin Rotated Loadings of the Six Factor Solution Item 1 2 3 4 5 6 OE1 0.673 0.075 0.079 0.018 0.031 0.015 OE2 0.741 0.031 -0.019 0.029 0.045 0.007 OE3 0.543 0.184 0.024 0.035 -0.023 0.151 OE4 0.450 0.152 0.256 0.046 0.034 -0.033 OE5 0.605 -0.044 0.064 0.044 0.180 0.057 SA1 0.238 0.387 0.007 0.025 0.056 0.032 SA2 0.012 0.658 0.133 -0.063 0.046 -0.033 SA3 0.073 0.499 0.134 -0.029 -0.003 -0.065 SA4 0.408 0.460 0.000 -0.021 0.040 0.006 SA5 0.173 0.496 0.027 0.139 -0.110 0.035 C1 0.069 -0.056 0.805 -0.005 0.028 -0.014 C2 0.147 0.019 0.663 -0.001 0.019 -0.037 C3 0.249 0.061 0.306 0.091 0.068 0.027 C4 0.008 0.166 0.558 0.004 -0.061 0.124 C5 -0.178 0.213 0.597 0.085 0.056 0.046 LDS1 0.054 0.039 -0.022 0.408 0.543 -0.122 LDS2 -0.121 0.060 0.076 0.161 0.601 0.027 LDS3 0.201 -0.021 0.004 0.001 0.601 0.147 LDS4 -0.006 -0.012 0.055 0.823 -0.002 0.040 LDS5 0.084 -0.009 0.049 -0.050 0.496 0.241 ODC1 0.091 -0.035 0.068 0.723 0.013 0.001 ODC2 0.010 0.250 -0.106 0.328 0.063 0.213 ODC3 0.145 -0.023 0.324 -0.029 0.002 0.437 ODC4 0.055 0.027 0.167 0.065 0.163 0.421 ODC5 -0.033 0.291 -0.028 0.077 0.029 0.416

Table 22 Geomin Rotated Loadings of the Five Factor Solution Item 1 2 3 4 5 OE1 0.740 0.055 0.038 0.001 0.008 OE2 0.808 0.018 -0.038 0.011 -0.014 OE5 0.723 -0.019 0.042 0.045 0.079 SA2 -0.018 0.675 0.130 -0.025 0.003 SA3 0.079 0.608 0.034 -0.009 -0.089 SA5 0.084 0.509 -0.029 0.125 0.071 C1 -0.001 -0.015 0.905 0.038 -0.040 C2 0.125 0.114 0.546 0.015 0.064 C4 -0.012 0.209 0.451 0.016 0.160 LDS1 0.282 0.061 0.028 0.457 0.018 LDS4 -0.041 0.011 -0.015 0.867 0.047 LDS6 0.080 -0.027 0.042 0.772 -0.039 ODC3 0.128 -0.020 0.250 -0.042 0.503 ODC4 0.183 0.006 0.104 0.050 0.475 ODC5 -0.035 0.244 -0.060 0.062 0.473

were all above the .4 criterion level which is commonly used in judging factor loadings as meaningful (Hinkin, 1998). Alpha was calculated using Stata 13. All but SA and ODC were above the recommended .70 cut-off in Hinkin (1998), however they were close at .678 and .68, respectively. Item reliability is subsequently addressed in the CFA section.