This section presents ANCOVA analysis (between groups), the paired sample t-test and Wilcoxon signed rank tests (within groups), in order to examine the effects of blending AOD with FTF learning on students‘ reading comprehension scores for each of the three literal, inferential, and evaluative levels.
5.2.1 Effect of BL on literal comprehension
Findings revealed that the adjusted mean of the BL group in post-test literal scores was higher than the adjusted mean of the control group (Table 10). The ANCOVA results indicated that the pre-reading literal scores (covariate) were significantly related to the post- reading literal scores, F(1, 61) = 10.61, p < 0.05, with a large effect size, ηp2 = 0.148.
However, the ANCOVA results revealed that there was no significant effect of BL on Paired Differences
M SD SD. Error t df Sig. (2-tailed) Pair 1 overall pre-test
literal scores, F(1, 61) = 0.49, p > 0.05, with a small effect size, ηp2 = 0.008. Thus, integrating
AOD with FTF reading instruction did not significantly improve students‘ literal comprehension scores compared with FTF learning (Appendix K).
Table 10
Descriptive Statistics for Pre and Post-test Literal Comprehension Scores
Pre-test Post-test
Groups n M SD M SD Adjusted mean
Experimental (BL) 32 3.78 1.01 4.47 0.76 4.46
Control (FTF) 32 3.69 1.15 4.31 0.82 4.33
Note. Maximum literal score = 5.
5.2.2 Comparison of literal comprehension scores before and after the experiments for both groups
Table 11 presents a Wilcoxon signed rank test applied to examine whether there was a significant difference between the pre- and post-test literal scores for the control group. The results of Wilcoxon tests revealed that there was a significant difference in literal comprehension scores from pre-test (Mdn = 4) to post-test (Mdn = 5), z = –2.56, p < 0.05, r = 0.32. These results suggest that the students who participated in the FTF group demonstrated a significant improvement from pre-test to post-test for literal comprehension scores.
Table 11
Comparison of Literal Comprehension Scores Before and After for the FTF Group: Wilcoxon Signed Rank Test
As shown in Table 12, the same test was applied to the pre-test and post-test literal scores for the BL group, revealing a significant difference on literal comprehension scores from pre-test (Mdn = 4) to post-test (Mdn = 5), z = –3.62, p < 0.05, r = 0.45. These results
z Asymp. Sig. (2-tailed) Literal pre-test compared with post-test scores (FTF) –2.561 0.010
suggest that the students who participated in the BL group demonstrated a significant improvement from pre-test to post-test for literal comprehension scores.
Table 12
Comparison of Literal Comprehension Scores Before and After for the BL Group: Wilcoxon Signed Rank Test
5.2.3 Effect of BL on inferential comprehension
As shown in Table 13, the adjusted mean of the BL group scores was higher than the adjusted mean of the control group scores in the post-test inferential scores. The ANCOVA findings indicated that pre-reading inferential scores (covariate) were significantly related to post-reading inferential scores, F(1, 61) = 28.49, p < 0.05, with a large effect size, ηp2 =
0.318. However, the ANCOVA results showed that there was no significant effect of BL on students‘ post-test inferential comprehension scores after controlling for the effect of pre-test inferential scores, F(1, 61) = 0.83, p > 0.05, with a small effect size, ηp2 = 0.013. These
results indicate that students who used AOD in combination with FTF classes did not improve significantly in inferential comprehension post-test scores compared with the students who used only FTF instruction (Appendix L).
Table 13
Descriptive Statistics for Pre and Post-test Inferential Comprehension Scores
Pre-test Post-test
Groups n M SD M SD Adjusted mean
Experimental (BL) 32 5.22 1.77 5.78 1.41 5.70
Control (FTF) 32 4.87 1.52 5.34 1.43 5.43
Note. Maximum inferential score = 8.
z Asymp. Sig. (2-tailed) Literal pre-test compared with post-test scores (BL) –3.620 0.000
5.2.4 Comparison of inferential comprehension scores before and after the experiments for both groups
Table 14 presents results of the paired sample t-test comparing pre- and post-test inferential comprehension scores for FTF. The results revealed that there was no significant difference on inferential comprehension scores from pre-test (M = 4.87, SD = 1.52) to post- test (M = 5.34, SD = 1.43), t(31) = –1.63, p > 0.05, r = 0.28. These results suggest that the students who participated in the FTF group did not demonstrate a significant improvement from pre-test to post-test for inferential comprehension scores.
Table 14
Comparison of Inferential Comprehension Scores Before and After for the FTF Group: Paired Sample t-Test
Table 15 presents the paired sample t-tests of the comparison pre-test and post-test of inferential comprehension scores for the BL group. The results revealed that there was significant difference on inferential comprehension scores from pre-test (M = 5.22, SD = 1.78) to post-test (M = 5.78, SD = 1.41), t(31) = –2.56, p < 0.05, r = 0.42. These results suggest that the students who participated in the BL group demonstrated a significant improvement from pre-test to post-test for inferential comprehension scores.
Table 15
Comparison of Inferential Comprehension Scores Before and After for the BL Group: Paired Sample t-Test
Paired Differences
M SD SD. Error t df Sig. (2-tailed) Pair 1 inferential pre-test
compared with post-test (FTF) 0.47 1.63 0.287 –1.631 31 0.113
Paired Differences
M SD SD. Error t df Sig. (2-tailed) Pair 1 inferential pre-test
5.2.5 Effect of BL on evaluative comprehension
Findings revealed that the adjusted mean of the BL group in post-test evaluative comprehension scores was higher than the adjusted mean of the control group (Table 16). The ANCOVA results indicated that pre-reading evaluative scores (covariate) were significantly related to the post-reading evaluative scores, F(1, 61) = 4.68, p < 0.05, with a medium effect size, ηp2 = 0.071. However, the ANCOVA results showed that there was no significant effect
of BL on students‘ post-test evaluative comprehension scores after controlling for the effect of pre-test evaluative scores, F(1, 61) = 2.75, p > 0.05, with a small effect size, ηp2 = 0.043.
This indicates that students who used BL did not improve significantly in evaluative comprehension post-test scores, compared with the FTF learning group (Appendix M).
Table 16
Descriptive Statistics for Pre and Post-test Evaluative Scores
Pre-test Post-test
Groups n M SD M SD Adjusted mean
Experimental (BL) 32 2.31 0.82 2.84 0.99 2.84
Control (FTF) 32 2.28 0.85 2.44 0.98 2.44
Note. Maximum evaluative score = 5.
5.2.6 Comparison of evaluative comprehension scores before and after the experiments for both groups
As shown in Table 17, a Wilcoxon signed rank test was applied to examine whether there was a significant difference between the pre- and post-evaluative comprehension for the FTF group. The results of Wilcoxon tests revealed that there was no significant difference in evaluative comprehension scores from pre-test (Mdn = 2), to post-test (Mdn = 2), z = –0.63,
did not demonstrate a significant improvement from pre-test to post-test for evaluative comprehension scores.
Table 17
Comparison of Evaluative Comprehension Scores Before and After for the FTF Group: Wilcoxon Signed Rank Test
As shown in the Table 18, a Wilcoxon signed rank test was also applied to examine whether there was significant difference between the pre- and post-evaluative comprehension for the BL group. The results of Wilcoxon tests revealed that there was a significant difference on evaluative comprehension scores from pre-test (Mdn = 2), to post-test (Mdn = 3), z = –2.78, p < 0.05, r = 0.35. These results suggest that the students who participated in the BL group demonstrated a significant improvement from pre- to post-tests for evaluative comprehension scores.
Table 18
Comparison of Evaluative Comprehension Scores Before and After for the BL Group: Wilcoxon Signed Rank Test