The scoring schedule (see Chapter 6, Table 6.7), of which participants were unaware, is shown in Table 7.5. The raw scores for participants and the change in score between the tests are shown in Table 7.6.
Table 7.5 Scoring schedule
Response
Score
Wrong response
-1
Don’t know/no idea response
0
Partially correct response
1
Fully correct response
2
The identical pre-test and post-test had 22 questions (see Chapter 6, Section 6.1.5); therefore a score of -22 would be obtained if a participant gave wrong answers to all questions. A score of 44 would be obtained where a participant gave fully correct answers to all questions. The maximum possible change is 66 (from -22 to +44), which would be the case if a participant gave all wrong answers in the pre-test and then all fully correct answers in the post-test, or vice versa.
Table 7.6 Participants’ raw scores in the pre-test and post-test
Participants
P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14
Pre-test scores (x)
18 25 29 16 31 14 32 14 27 20 30 28
9
22
Post-test scores (y)
31 31 30 30 43 43 38 32 30 24 42 40 27 42
Change in understanding
(y) - (x)
13 6 1 14 12 29 6 18 3
4
12 12 18 20
The highest pre-test score of 32 was recorded by participant P7 and the lowest score of 9 was recorded by participant P13 (Table 7.6). The highest post-test score of 43 was recorded by two participants (P5, P6), while one participant (P10) scored 24, which was the lowest score (Table 7.6). The largest change in understanding between the pre-test and post-test was 29 and the smallest change, 1. Based on the comparison of raw scores, the data in Table 7.6 suggest that each participant increased their personal understanding about lake values after using the ElleVis tool.
To further ascertain the extent of an increase in understanding following use of ElleVis, summaries of all participants’ responses in the pre-test and post-test were compiled as shown in Table 7.7 and Table 7.8. The data in Table 7.7 and Table 7.8 were normalised to take account of the different number of sub-questions to each main question. Questions 1 and 2 had 10 sub-questions each relating to different lake values, and Questions 3 and 4 had one sub-question each, making a total of 22 questions. Normalisation involved multiplying the data in Questions 3 and 4 by a factor of 10 (since both questions had one sub-question each) to equate with the weightings of Questions 1 and 2 which both had ten sub-questions each.
Table 7.7 Participants’ normalised pre-test scores
PRE-TEST
Wrong answer
response Don't know response
Partially correct
response Fully correct response
Question 1 17 38 19 66
Question 2 17 21 16 86
Question 3 10 0 40 90
Table 7.8 Participants’ normalised post-test scores
POST-TEST
Wrong answer
response Don't know response
Partially correct
response Fully correct response
Question 1 9 6 31 94
Question 2 10 0 13 117
Question 3 0 0 40 100
Question 4 50 0 0 90
The mean scores (all participants) increased from 22.5 in the pre-test to 34.5 in the post-test. The standard deviation (all participants) decreased from 7.4 in the pre-test to 6.5 in the post-test. The higher post-test scores (Table 7.6) means that all participants increased their understanding about lake values after using the ElleVis tool. The results in Table 7.6-7.8 show that participants’ understanding of the impact of Te Waihora/Lake Ellesmere opening regimes on lake values increased after use of ElleVis. The next section presents the results of the statistical tests conducted on participants’ changes in personal understanding following their use of ElleVis. These statistical tests are necessary as the significance of the scores cannot be deduced from descriptive statistics.
7.2.1
Paired samples t-test for raw scores
A paired samples t-test (Bakan, 1966) was undertaken to test the statistical significance of the participants’ acquisition of understanding following their use of ElleVis. The test was undertaken to assess whether the group means of the pre-test and post-test were statistically different from each other. This was determined using a two-tailed ‘t’ test with an alpha of 0.05 and using a null hypothesis that the group means of the pre-test and post-test were equal.
Table 7.9 Paired samples statistics for raw scores
Mean N Std. Deviation Std. Error Mean
Pair 1 Post-test Score 34.50 14 6.549 1.750
Pre-test Score 22.50 14 7.408 1.980
Table 7.10 Paired samples t-test for raw scores
Paired Differences t df Sig. (2- tailed) Mean Std. Deviation Std. Error Mean 95% Confidence Interval of the Difference Lower Upper
Pair 1 Post-test Score ─
Pre-test Score
The results in Table 7.9 and Table 7.10 show that overall, participants’ scores were significantly higher (M = 34.5, SD = 6.549) after using the ElleVis visualisation tool than before (M=22.5, SD = 7.408), t (13) =5.842, p < .001.
The results in Table 7.9 and Table 7.10 show that participants’ personal understanding of the impact of Te Waihora/Lake Ellesmere opening regimes on lake values increased significantly after use of ElleVis. To test whether the group medians between the pre-test and post-test were statistically different from each other, an additional test – a Wilcoxon signed-ranks test – was conducted. The Wilcoxon signed-ranks test is best suited for comparison of median differences when the distribution between the pairs (pre-test and post-test scores) cannot be assumed to be normally distributed (Bakan, 1966; Conover & Iman, 1981). The Wilcoxon signed-ranks test uses the null hypothesis that the median difference between the pairs of observations (i.e. pre-test and post-test scores) is zero.
7.2.2
Wilcoxon signed-ranks test for raw scores
The Wilcoxon signed-ranks test (Bakan, 1966) evaluates the statistical significance of the participants’ acquisition of understanding following their use of ElleVis, using a null hypothesis that the median differences between the pre-test and the post-test scores are equal to zero, with an alpha of 0.05.
Table 7.11 Wilcoxon signed-ranks test for raw scores
This test (Table 7.11), shows that overall, participants scores were significantly higher (Median =31.50) after using the ElleVis visualisation tool than before (Median = 23.50), Z= -3.301, p = .001, r =. 88.
The results given in Tables 7.9 ─ 7.11 therefore support the conclusion that participants' personal understanding of the impact of Te Waihora/Lake Ellesmere opening regimes on lake values significantly increased after use of ElleVis.
Having discussed changes in participants’ personal understanding of the impact of opening regimes on lake values, it is now important to consider the impact on their shared understanding of such values.
Ranks Test Statisticsb
N Mean Rank Sum of Ranks Post-test Scores – Pre-test Scores Post-test Scores ─
Pre-test Scores Negative Ranks 0a .00 .00
Positive Ranks 14b 7.50 105.00 Z -3.301a
Ties 0c Asymp.
Sig. (2-tailed) .001 Total 14 a. Based on negative ranks.
b. Wilcoxon Signed Ranks Test a. Post-test Scores < Pre-test Scores
b. Post-test Scores > Pre-test Scores c. Post-test Scores = Pre-test Scores