4.2.1
Design and Participants
To test the hypotheses, the first script study was conducted as a two-group between- subjects experiment with participants working in dyads. At the start of the labo- ratory experiment (t1), the sample consisted of N = 28 university students (14 fe- males, 14 males) with a mean age of M = 23.29 (SD = 5.72). All experiments were conducted in a research laboratory where participants and were seated at comput- ers separated by privacy blinds. They were randomly assigned to a BRD or DDR script group and received further instructions on the computer. Participants rated their topic-specific interest as medium to high (broader topic: M = 3.50, SD = 1.11; specific topic: M = 3.14, SD = 1.08) and their self-assessed prior knowledge about the topic ranged between low and moderate (broader topic: M = 2.00, SD = 1.02; specific topic: M = 0.25, SD = 0.80) (cf. 4.2.3 for information on the scales used). Between groups, there were negligibly small to no differences in topic-specific inter- est (broader topic: U = 62.00, z = -1.71, p = .104, r = -.32, 95% CI [-.62, .06]; specific topic: U = 73.00, z = -1.12, p = .265, r = -.23, 95% CI [-.55, .28]) or in prior knowl- edge (broader topic: U = 72.00, z = -1.27, p = .246, r = -.24, 95% CI [-.56, .15]; specific topic: U = 91.50, z = -0.56, p = .769, r = -.11, 95% CI [-.46, .28]). Two weeks after the
lab study (t2), participants were asked to take part in an online post-test about the study’s contents. N = 22 students (12 females, 10 males) accepted to participate in this post-test.
4.2.2
Procedure and Materials
After being welcomed and declaring their consent, participants were placed in front of a computer where they received further instructions to start the experiment. First, they were asked to provide demographic information (e.g. gender and age) and to rate their interest in and prior knowledge about the study’s controversial topic. The broader topic was pirates in general and more specifically a historical personality, namely the alleged pirate Captain William Kidd. Participants then read a wiki ar- ticle about the Scottish-American buccaneer William Kidd and supplementary ma- terial that was presented to the participants individually. In each learning dyad, one participant received (A) evidence that supported the theses that William Kidd was a pirate and his trial was justified and the other received (B) evidence against those theses. In a mandatory tutorial, they were then introduced to the phases and workflows of either the BRD or the DDR script. Detailed instructions were given as texts and accompanied by graphical representations of the most relevant phases of the scripts. Following this, representations of the collaboration scripts as work flow diagrams depicting the most relevant phases were permanently visible in the learning environment (cf. Figure 4.2). Learning partners were invited into a learn- ing environment with the collaborative online open source editor Etherpad. Prior to the experiment’s collaboration phase, participants had the opportunity to famil- iarise themselves with Etherpad and its functions in a prepared tutorial. The study’s main task in both groups was to edit a wiki article about William Kidd with the help of the supplementary materials each participant received.
Figure 4.2. Wiki-mimicking learning environment with collaboration script representation in Experiment 3.
In the collaboration phase a basic article about William Kidd was preloaded into Etherpad. This article was structured in a way to mimic a standard Wikipedia article and had a total length of 810 words. Because the lab experiment was constrained in time, Etherpad’s built-in chat functionality was activated by default and worked as a substitute for the corresponding wiki article talk page. After the collaborative editing task, participants were again asked to answer questions about their interest in the study’s topic and the likelihood that they will search for more information af- ter the experiment. This was followed by a short questionnaire on their individual need for cognitive closure. As an immediate test of what participants had learned about the topic, a knowledge test with multiple choice and open-ended questions
was deployed at the end of the session (t1). Two weeks later participants were in- vited to participate in a second knowledge test (t2) with a different but comparable set of multiple choice and open-ended questions. Figure 4.3 shows the overall study procedure.
Figure 4.3. Workflow diagram visualising the overall study procedure of Experiment 3 with its central stages.
4.2.3
Variables and Measurements
The main independent variable was the random allocation to either of two experi- mental wiki-like environments proposing different collaboration scripts. While par- ticipants of one group were assigned to the collaborative editing workflow proposed by Wikipedia (BRD), the other group was assigned to the script that was designed with a stronger focus on upfront discussion and deliberation processes (DDR).
The learning outcomes were measured at two points in time, immediately at the end of the experiment (t1) and two weeks later with an invitation-only online ques- tionnaire (t2). This leads to a 2 x 2 mixed factorial design with one between- and one within-subjects factor with individuals as unit of analysis, ICC = -.07, F (13, 14) = 0.87, p = .598. In addition to socio-demographic variables (e.g. gender and age), topic-specific interest and prior knowledge about the study’s topic were as- sessed. Participants were asked to provide self-ratings on six-point scales ranging from “not interesting at all” / “no prior knowledge” (0) to “very interesting” / “high prior knowledge” (5). To measure the impact either collaboration script had on the knowledge construction processes and learning outcomes, wiki contribution metrics were recorded by the wiki environment and two knowledge tests were deployed. Contributions to the wiki were analysed in terms of article length, discussion length and the number of ideas from the supplementary materials that were incorporated into the original article. Measures of article and discussion length functioned as quality indicators, but also as manipulation checks to infer if participants collab- orated in accordance with the respective script. Between the collaborative writing task and the first assessment of learning, the personal need for cognitive closure was measured with the German short scale 16-NCCS. The two knowledge tests to mea- sure the learning outcomes each comprised ten multiple choice questions with four alternative answers and two open-ended questions. In each of the multiple-choice tests, a maximum score of 17 correct answers could be achieved. In the open-ended questions participants were asked if they think that (1) Captain William Kidd was a real pirate and (2) if his conviction was justified. These questions were used to assess whether a participant provided only arguments presented in the individual supple- mentary material, or if they gave more differentiated statements by integrating the evidence from their partner’s material.