Asociaciones biológicas

students did not attend to the signal–noise idea and that their natural reasoning pro- cesses did not entertain effects of randomness and chance.

In the posttest, responses from students conveyed more of an appreciation of the ideas encompassed within the notion of experimental design and causation with fewer concerns about representativeness, group size or confounding variables within Action 1. However, the signal–noise idea still needs to be further developed. Most of the chance explanations provided by these students and the wider cohort did not appear to convey the understanding that chance is always acting. Instead, the overrid- ing impression was that most students had the underlying notion that either chance is acting alone, or chance is not acting at all. Such a problem is not surprising given that students only had two hours of instruction. However, we need to be aware that further instruction should address these two apparent conceptions:

If the evidence favors a chance alone explanation, it excludes the possibility that the treatment may be effective.

If the evidence favors a treatment explanation, it excludes the possibility that chance is acting alongside treatment.

Thus the idea that “treatment is effective” comprises both treatment and chance com- ponents, and the idea that “chance is acting alone” does not rule out the effectiveness of the treatment, is a learning issue that needs to be addressed. We believe that such conceptions can partly be attributed to the logic of the indirect argumentation asso- ciated with making a claim as a result of experiment-to-causation inference.

4.5.2 Action 2: Modeling Random Behavior

For Action 2 the students referred to Questions 1 and 2 (see Appendix). They were asked about the purpose of the test and to recall the randomization test in order to de- termine their understanding of how the distribution in Question 2 was formed. Note that Questions 1 and 2 were not presented in the format and with the representations used in the dynamic visualizations (for a comparison see Figure4.1). Hence, the students needed to decode the representations given and recall the re-randomization process. To quantify the uncertainty on the distribution given in Question 2, the stu- dents needed to take the difference in means given in the table and plot it. Unlike the VIT software which gives the tail proportion visually and numerically when a button is activated, they had to recall the observed difference being plotted, the tail propor- tion being shaded in and then work out that they had to roughly count the number of differences equal or greater than 7.71. Before we elaborate on the student responses, an overall summary of the elements of reasoning and ideas that we were looking for is given in Table4.4along with student examples, codes, and descriptors for each of the elements.

Focusing on the responses of S1 to S4, we use the codes T1 to T5 (see Table

4.4) to illustrate how the randomization test was promoting ideas of uncertainty and where more development in students’ reasoning appears to be needed.

ANALYSIS AND RESULTS 111

S2 could state that the randomization test was determining “the chance of getting the result that we did in the circumstances where chance could be acting alone (T1)” and that there was a “mixing up of the conditions with the observations if everything was due to chance (T2).” He knew that each dot in the distribution represented a dif- ference between the means (T3), that the process was repeated 1000 times and that a distribution developed (T4). However, his reasoning within element T4 faltered as he failed to connect that chance acting alone was visually represented by the distribu- tion and he said he was “confused.” He was also unable to obtain the tail proportion (T5). The interviewer asked, “so then what would be the observed difference if you were to plot that from this graph [points to graph in Question 1].” He responded, “I assume 7.71, oh right, it would be about there [he locates 7.71 on the distribution and puts a box around the tail proportion], yeah I’ve got it now.” At that moment he connected the steps in the procedure for the randomization step, found the tail pro- portion and quantified the uncertainty (T5). Thus we conjecture S2 had a fragmented understanding of the randomization test process. He is developing ideas of a repeated chance process forming a distribution but is not yet fully connecting the underlying concepts.

S3 was able to succinctly describe the purpose of the randomization test: “You have a measurement of the difference and with the randomization test you measure how likely it is that chance alone will produce the same difference (T1).” He then followed with a description of the randomization test process.

They separated the results from the group and then just randomly assigned them in a resample (T2), and then they took the mean difference of that (T3) and then repeated that process a 1000 times in this case, and it’s got a distribution of what was possible by chance alone (T4), and then compared the result that they got from the actual test with the distribution, to get a tail of how likely it was (T5), if it was just chance.

For S3 one of the dots in the distribution “would represent chance.” It is “just one difference between the means for a re-sampling.” Hence unlike S2, he seems to make the connection between the notion of chance alone and that the distribution is a visual representation of chance alone. He was also able to quantify the uncertainty by putting 7.71 on the distribution and calculating the tail proportion, as did S1 and S4.