Reliability and validity analysis is the first step in the questionnaire analysis, and it is also one of the criteria for checking whether the questionnaire is qualified. The analysis of the questionnaire is reliable only if the reliability and validity are high. The method of reliability and validity analysis includes logical analysis and statistical analysis. Logical analysis is to select the test questions based on a theoretical basis or reasoning. The scale used in this thesis has verified by researcher, so logical analysis would not be included. Statistical analysis means that questions in the same scale should be highly related and consistent. Although reliability and validity of these scales have been tested previously, but it is necessary to analyze whether they are applicable to our research subjects.
3.6.1.1
Reliability analysis
Reliability refers to the degree of consistency in the results obtained by repeatedly measuring the same object using the same method. It reflects the internal consistency or stability of the measuring tool. The methods of reliability analysis mainly include retest reliability, parallel-forms reliability, split-half reliability and α confidence coefficient method. Through the reliability test we can understand the reliability and
95
effectiveness of the questionnaire results.
In the absence of repeated measurements and parallel-forms, Cronbach's alpha is generally used as predictor of reliability (Xiufeng Ding, 2001). The Cronbach's α is also called the internal consistency coefficient, which examines the consistency of various questions within the test, i.e. whether the questions have same or similar measurement goals and whether they have the same or similar psychological features. The higher the internal consistency coefficient, the smaller the random error of the measurement. Thus, the measured results are more consistent with high reliability. When the internal consistency coefficient of a project is greater than 0.70, it indicates that its internal consistency is good. If the Cronbach's α coefficient reaches 0.8 or higher, the consistency of the measurement results is excellent. If it is more than 0.9, there may exist redundant items.
In addition, the reliability analysis can also be used as a basis for the deletion of the questionnaire questions. If total correlation coefficient is less than 0.3 after removing the question item and the overall reliability of the scale has significantly improved, then it is reasonable to consider removing the question.
3.6.1.2
Validity analysis
Validity is effectiveness, which refers to the degree of how measurement tools or approaches can accurately measure subjects to be measured. Validity is the most important requirement for scientific measurement. The validity types include Face Validity, Content Validity, Construct Validity, and Criterion Validity. The face validity refers to judging the accuracy of the test by common sense or subjective assumptions, mainly focus on how measurement and procedure proceeds. Content validity refers to the degree of how measurement tool reflects how researcher measures the subjects. Construct validity refers to the mapping between the theoretical construct and the measured value reflected by the measurement results. The criterion validity refers to the consistency of criterion and test results, which is also referred to as principle validity or predictive validity. This thesis focuses on the content validity and construct validity
96
of the questionnaire.
The determination of content validity is a process of reasoning and judgment, with no quantitative indicator available. To have higher content validity, the measurement must meet two requirements: the first step is to determine the scope of content and contain all questions within this scope. The next requirement should be representative samples within the defined content scope, i.e. the selected items can represent the main aspects of the tested content scope and each item is proportionally appropriate. Construct validity includes Convergent Validity and Discriminant Validity. Convergence validity refers to the relatedness of the item and the construct. Discriminant validity refers to the un-relatedness of the item and the construct. The most desirable method of construct validity analysis is to use factor analysis to measure the construct validity of the scale or the entire questionnaire. The main purpose of factor analysis is to draw some common factors from all the variables (items) of the scale. Each common factor is highly correlated with a particular group of variables, which represent the basic construct of the scale. Through the factor analysis, it is possible to examine whether the questionnaire can measure some of the constructs assumed by the researcher while designing the questionnaire. In factor analysis, the main indicators used to evaluate construct validity include cumulative contribution rate, communalities and factor loading. The cumulative contribution rate reflects the cumulative validity of the common factor to the scale or questionnaire. The communality reflects the validity of interpreting the original variable by the common factor. The factor loading reflects the variance between the original variable and the common factor.
3.6.1.3
Factor analysis
Factor analysis is the extension of principal component analysis, which is a multi- variable statistical analysis method drawing a few factors out from variables with intricate relations based on the dependency within matrix. Factor analysis can discover hidden representative factors in many variables. Variables of the same nature are classified into one factor, which reduces the number of variables and verifies hypothesis
97
on the relationship between variables. Factor analysis can be divided into exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). Exploratory factor analysis and confirmatory factor analysis are two stages of the research process, which shall be integrated rather than separated to conduct in-depth research.