The internal consistency of the items chosen for this study was determined using the Cronbach alpha. The Cronbach alpha values for each subscale were calculated and found to be as follows; Asking/framing research questions (α = 0.81), designing investigations (α = 0.75), conducting investigations (α = 0.77), collecting data (α = 0.83), and drawing conclusions (α = 0.77). As mentioned earlier, a Cronbach alpha as low as 0.55 can still be recognized and accepted for statistical consideration (Hatcher & Stepanski, 1994). Considering that no Cronbach alpha value was found below 0.55, it signifies that the results indicate that the instrument achieved a satisfactory level of internal consistency. An overall Cronbach alpha value of 0.79 was obtained. The actual PSI-S questionnaire responses used in the production of the instrument produced an overall Cronbach alpha of 0.64 (Campbell, et al., 2010) for the whole instrument.
4.3 Discussion
In this section, results from the pilot study are discussed. This discussion is organized under the LUSSI instrument and the LPCI instruments respectively.
170 4.3.1The LUSSI instrument
Evidence relating to criterion and construct validity of the LUSSI instrument is provided by the findings of the pilot study. Results from multiple regression analysis show that five out of six variables correlate substantially with the dependent variable and because one variable (change of scientific theories) did not substantially correlate with the dependent variable, SC Total, it was dropped before running the collinearity diagnostics. Collinearity diagnostics was performed on the five remaining dimensions of the instrument since they had significant correlation with the dependent variable as part of the multiple regression procedure. No problems were found with multicollinearity. Assumptions like outliers, normality, linearity, homoscedasticity and independence of residuals were checked and no violations to the assumptions were found.
The pilot study has provided some evidence relating to face, content, criterion and construct validity of the LUSSI instrument as used in the South African context. Both professional science educators and post-graduate students agreed that the instrument items aim to solicit from learners’ their views on the processes of science and how science develops, i. e. their conceptions of the nature of scientific inquiry. These results are consistent with the findings of Liang et al. (2006) who interviewed professors of science education from China, USA, Taiwan and Turkey and found the SUSSI have face and content validity. If these findings are combined with results form both criterion and construct validation and internal consistency results where a relatively high Cronbach alpha (0.80) value was obtained in the pilot study one can safely and confidently recommend the use of this instrument in large scale quantitative studies on the African continent. But at the same time suggesting that researchers and professional developers using the instrument in the future still pilot the instrument as the tenet (changes of scientific theories) did not correlate with others before running multiple regression analysis.
171 4.3.2 The LPCI instrument
This study has provided some evidence relating to face, content, criterion and construct validity of the LPCI instrument as used in the South African context. Contrary to the researcher’s expectations, results from interviews with expert science educators and the learners themselves point towards the instrument having face, content and construct validity even within the context of the South African science classroom. Like what other researchers did (e.g., Angoff, 1988; Cronbach & Quirk, 1976; Li, 2003) the researcher investigated the fit between the conceptual definitions and operational definitions of variables in the LPCI qualitatively. Interview results appear to give this instrument a thumps up. If these findings are combined with the relatively high Cronbach alpha (0.80) value obtained in this pilot study- based on the sample, one can safely and confidently recommend the use of this instrument in large scale quantitative studies on the African continent. The findings from learner interviews are consistent with previous work by Campbell et al. (2010) with 130 secondary science students in the, USA. Their results support the face validity of the LPCI. The fact that the LUSSI produced these results with South African learners is a surprising finding. This was unexpected given the poor language proficiency of South African secondary school learners (Linneman, et al., 2003).
From the two focus group interviews held with the learners we can conclude that the LPCI instrument appears to have both face validity and to be written in a language which the learners can understand. "On its face", the instrument seems like a good translation of the construct (McGartland & Kimberly, 2005; Miller, 2009). Overall, from the discussions held with the experts one can “safely say” the instrument shows content and construct validity. Taking the above responses into account, the instrument can be said to measure what it is suppose to measure.
Results from multiple regression analysis show that five variables correlate substantially and because correlation is substantial, no variable was dropped before running the collinearity diagnostics. Collinearity diagnostics was performed on the five categories of
172
the instrument since they had significant correlation with the dependent variable as part of the multiple regression procedure. No problems were found with multicollinearity. Assumptions like outliers, normality, linearity, homoscedasticity and independence of residuals were checked and no violations to the assumptions were found. Since criterion validity is about prediction rather than explanation and is concerned with non-casual or mathematical dependence, we can safely say, the multiple regression analysis results provide evidence relating to criterion validity of the LPCI instrument. This is consistent with findings from other studies (e.g., Heppner, et al., 1992; Meyer, Miller, Metzger, & Borkovec, 2002; Sullivan & Karlsson, 1999) where Regression analysis was used to validate research instruments.
In explaining how much variance in learners’ perceptions of classroom inquiry could be explained by scores on the five facets of scientific inquiry, the model (which includes the five variables) explained 100 per cent of the variance in Total Score. The model also reached statistical significance (Sig .000, meaning p <.0005) meaning result is statistically significant. These results show that both criterion and construct validity were tested for in the instrument through use of multiple regression as done in other studies (see, Arozullah, et al., 2007; Osborne, 2000). From the results the pilot study recommends the use of multiple regression analysis as a methodological technique in predicting criterion validity of phenomena related to learners’ experiences of scientific inquiry.
Internal consistency results show that all five processes of science in the LPCI instrument correlated very well with high Cronbach alpha values obtained. These results are consistent with findings from the study by Campbell et al. (2010) when they piloted the same instrument. A decision was made to maintain the five original categories of the instruments in the final version of the instrument used in the main study. However, a recommendation made is that researchers and professional developers using the instrument in future be cognisant of internal consistency of instrument items as potential measures of progress toward reform in implementing inquiry as an instructional strategy.
173
From the interview results above, the interviewees’ responses point to the fact that the LPCI was found to be face and content valid. Responses also point in the direction that the LPCI instrument purports to measure what it is intended to measure, that is, learners’ perceptions of the extent they practice inquiry in their science classroom. Thus the instrument was found suitable for use in the South African classroom.
4.4 Conclusion
Both the investigated instruments were found to be suitable for use in the main study. The confidence to use the instruments came from the results of both multiple regression analysis and interviews. In terms of face, content, criterion and construct validity, the two instruments can be said to have passed the test. Interviews (individual and focus group interviews) provided rich qualitative data that made it possible to conclude about the validity of each instrument. However, based on the pilot findings, the performed inferential statistics suggest that the tenet ‘change of scientific theories’ on the LUSSI instrument may be dropped from the instrument to ensure that the remaining tenets correlate. However, it was retained in the instrument for the main study because the participants appeared to be more informed about the tentativeness and durability of theories in their open-ended responses.
The results of this pilot study are significant in that they provided guidance about the sample sizes for the main study. The pilot also helped in shaping the sampling procedure and techniques for the main study. Furthermore, the results from the pilot invite thought about how NOSI views and learner perceptions of the extent of inquiry in science classrooms can be measured in larger populations. They also provide insights into the complexity and challenges involved in measuring and interpreting learner NOSI views and learner perceptions of the levels of classroom inquiry.
174