The measurement of spatial ability is suggested to be one of the most influencing prerequisites to learn with multimedia presentations, prior knowledge is another important learning prerequisite that has to be measured in every learning study. The following section introduces the spatial ability and prior knowledge test used in all three experiments.
Spatial Ability. The prerequisite spatial ability is tested by two subtests paper-folding and card-rotation adapted from the standardized tests summarized in the manual for kit of factor- referenced cognitive tests (Ekstrom et al., 1976). The experimenter introduces these tests to the participants by describing the goal of measuring how well they can imagine something in its spatial form. First, the test of card-rotation is introduced by reading the written instruction aloud, which includes an example task. Participants are prompted to check every given rotated card out of eight cards per line on its fittings with the original card given at the beginning of the line. Participants have to imagine the rotation of the original card to check the similarity to the card in question. After the solution of the example task is given, participants have time to correct their answer and/or to ask questions. Then, the experimenter signals to turn the page in order to begin the card-rotation test determined by the overall 80 card-rotation items within the given 3 minutes. The experimenter signals the end of the restricted time and introduces the paper-folding test. This test is also given by reading the written instruction aloud and solving an example task. Here, five solution papers are given per line and the only correct one should be discovered that fits with the folded and hole-punched paper presented at the beginning of the line. Participants have to imagine the folding action and the consequences for the initially punched holes. After the solution of an example task is given, participants have time to correct their answer and/or to ask questions. Then, the experimenter signals to turn the page in order to begin the paper-folding test limited by overall 10 paper-folding items within the given 3 minutes. The experimenter signals when the restricted time is over. The spatial ability value a participant reaches is determined by the average of both test values, the mean of the percentage of correct answers in the card-rotation test and the percentage of correct answers in the paper-folding test.
Prior knowledge. For the diagnose of the domain-specific prior knowledge, a computer-based pretest with five multiple choice and eight open-response formatted questions about the content domain is introduced in all experiments at the beginning of the learning program. The construction of this pretest considers different levels of prior knowledge including questions
about structural and procedural knowledge. Table 13 shows all items of the prior knowledge test. Every question can be answered by using the option “I don’t know” instead of formulating an answer to the open-ended questions or choosing the already formulated multiple choice answers.
Table 13. Prior Knowledge Test Items
Task Required Theoretical
Level of Knowledge Maximum
1. Do you know the term “Synthese”? If yes, then
please write down its meaning (catchwords)! [ORF] factual knowledge 1 2. Do you know the term “ATP”? If yes, then
please write down its meaning (catchwords)! [ORF] factual knowledge 1 3. Do you know the term “ADP”? If yes, then
please write down its meaning (catchwords)! [ORF] factual knowledge 1 4. Do you know the term “proton canal”? If yes, then
please write down its meaning (catchwords)! [ORF] factual/comprehension 1 5. Do you know the term
“conformation modification”? If yes, then
please write down its meaning (catchwords)! [ORF] comprehension 1 6. Name the organelle of the cell respiration! [ORF] factual/comprehension 1 7. Which main function does the ATP have? [MC] comprehension 1 8. Proteins are often labeled in reference to their
Function. Which function does a protein have
that is labeled “Synthase”? [MC] comprehension 1 9. The ATP-Synthase is a … [MC] comprehension 1 10. Which task performs the ATP-Synthase? [MC] comprehension 1 11. Of which two subcomponents is the
ATP-Synthase composed? [ORF] factual knowledge 1 12. Which description is the best one for the
molecular structure of the ATP-Synthase? [MC] factual knowledge 1 13. Shortly describe, what is meant by a
proton gradient! [ORF] factual/comprehension 1 Note. ORF = Open Response Format; MC = Multiple Choice.
In order to answer each of these questions, a basic knowledge of the issue ATP-Synthase is necessary. The theoretical maximum is one point per item and for the complete test 13 points. Some of the items are associated with factual knowledge, while others are demanding a higher level of knowledge, which is labeled comprehension. The first three questions only ask for definitions, which could be remembered without the necessity to understand them. Questions 11 and 12 also ask for pure factual knowledge, because these items are about the structure of the molecule. All other items require a higher level of knowledge to be able to answer correctly. This is indicated by the word comprehension in Table 13, as the issue has to be understood in more detail to know the right answer. An analysis sheet for all items including examples of correct answers to questions with open response format is used to get a high interrater reliability. The intercorrelations of the 13 test items are illustrated in Table 14 from Experiment 1 to get an impression of these pretest items used in all experiments.
Table 14. Intercorrelations of Prior Knowledge Test Items
Note. * p < .05, ** p < .01, two-tailed significance testing.
Prior knowledge test items correlate with each other on very different levels, illustrated by the varying correlation coefficients in Table 14. Accordingly, the reliability of the prior knowledge test presents a cronbachs alpha of .70. As it was the goal to construct a heterogeneous test, which measures different knowledge aspects and levels not necessarily referring to each other, the reliability of this test is not as high as it could be estimated its (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) (13) (1) Item 1 .13 .09 -.08 .19 .16 .02 .27** .26** .23* .16 .16 .06 (2) Item 2 .77** .19 .12 .18 .03 .05 .20* .09 .07 -.14 .12 (3) Item 3 .21* .13 .28** .16 .15 .25* .16 .08 -.14 .28** (4) Item 4 -.02 .15 .03 .09 .18 .17 .17 -.06 .28** (5) Item 5 .01 .07 -.03 .30** .07 -.02 -.02 -.10 (6) Item 6 .26** .32** .36** .18 .13 -.08 .38** (7) Item 7 .18 .18 .16 .04 .04 .24* (8) Item 8 .24* .33** .11 .11 .23* (9) Item 9 .15 .23* -.04 .10 (10) Item 10 .18 -.06 .27** (11) Item 11 -.01 .20* (12) Item 12 -.06
internal validity. It should be mentioned that this test of reliability underestimates the quality of the present prior knowledge test, because the internal consistency is only representing the homogeneity of a test and thereafter underestimates heterogeneous test constructions (Bortz & Döring, 1995). In addition, the reliability of the prior knowledge test: In Experiment 2 and 3 cronbachs alpha reached a value of .62 and .74, respectively and all data together of all three experiments result in a cronbachs alpha of .74.