Users need to make many decisions during the procedure of the unified approach. First, they need to pick specific types of misspecification and corresponding level of maximal trivial misspecifi- cation. Second, they need to pick the alpha level, which subsequently determines the confidence level of the confidence intervals. Users also need to select the fit indices to be included in global fit evaluation. The standards for the last two decisions can be easily made. Selecting the standards for the type of misspecification and the level of maximal trivial misspecification, however, is not an easy task.
The types of misspecification may vary models. Although I have reviewed several types of misspecification (factor loadings, measurement error correlation, factor correlation, or regression coefficients), they are applicable to relatively simple models (e.g., confirmatory factor analysis). These types of misspecification are also by no mean exhaustive. For example, the differences be- tween factor loadings within a construct is a potential source of misspecification in a tau-equivalent model, which are not included in the past research. To make the result more generalizable, it is recommended to account for sources of misspecification as many as possible. To do so, researchers may start with the list of modification indices by treating all fixed parameters as sources of mis- specification. Then, they can consider other misspecifications besides the fixed parameters. For example, an extra factor with small variance may be included as a trivial misspecification in a CFA model.
For all types of misspecification, researchers need to set the level of maximal trivial misspecifi- cation. The level of maximal trivial misspecification could be different across substantive areas. As shown in Chapter 3, I reviewed several types of misspecification considered in the past research. Some studies suggested guidelines on the level of misspecification. I reviewed the range of values that are considered trivial or severe misspecifications in the previous studies. Unfortunately, there is no consensus on these values. For example, either .1 or .3 may be used as the threshold for maximal trivial misspecification for measurement error correlations. Because of the subjectivity of the decision, there is a chance that researchers will abuse it by picking the value that would lead to a preferred result. However, the problem is not unique to the unified approach but also exists in the one-size-fit-all cutoffs. For instance, researchers may pick the CFI cutoff of .90 (Bentler & Bonett, 1980) or .95 (Hu & Bentler, 1999) to get their desired results. Even though this problem from one-size-fit all cutoffs remains in the unified approach, the parameters values that researchers used to specify the level of maximal trivial misspecification are more meaningful than specific val- ues of fit indices. For example, researchers know the amount of misfit by specifying the level of misspecification at cross loadings (e.g., not greater than .3). In contrast, researchers do not know the amount of misspecified cross loadings for a given value of a fit index because fit indices are influenced by model characteristics (e.g., the number of items or the amount of target factor load- ings). Alternatively, researchers may set the level of maximal trivial misspecification by examining whether the results from misspecified models provide accurate parameter estimates. This issue is discussed in the last section of the limitations of the unified approach.
On the one hand, to set up a standard for model evaluation in structural equation modeling, a consensus needs to be reached in using the unified approach in terms of the common types of misspecification and level of maximal trivial misspecification for a specific type of model. The standard will facilitate the communication among researchers when a well-fitting or bad-fitting model is claimed. On the other hand, different substantive areas can have different levels of maxi- mal trivial misspecification. A high-risk study may set up a stricter standard on specifying the level of maximal trivial misspecification. For example, researchers may specify the error correlations of
.2 as the maximal trivial misspecification for a scale used in correlational research. However, if a scale is used for individual assessment (e.g., intelligence test), the level of maximal trivial misspec- ification for error correlation can be lower (e.g., .05). Omitted error correlations of .2 may lead to slightly distorted target factor loadings and factor scores for each individual. The distorted factor scores can change the category of diagnosis (e.g., from low average to borderline in intelligence tests).
Based on the discussion above, there is not a good way to solve the subjectivity problem be- cause a single standard cannot be applied to all substantive areas. However, to add more objectivity into the procedure, I recommend that a standard should be established for the maximal trivial mis- specification in each substantive area. Once the standard is established, justification is required if a researcher proposes to use values different from the standard. This information should be trans- parently reported to readers. If readers disagreed with the proposed values, they may reanalyze the data with different sets of maximal trivial misspecifications. Furthermore, researchers may imple- ment the unified approach by two or more sets of maximal trivial misspecifications. This practice will show how the conclusion is sensitive to different sets of maximal trivial misspecification.