It was decided to explore OpenSim with the next student cohort for the same HCI course module student projects as an alternative MUVE platform. For the projects, students were asked to develop an interactive door system for an enclosure with two doors minimum: one from simple touch-open, access key based or password based, and the other from complex arrangements such as keypad entry, combination access and fire escape door with alarms, etc. A dedicated region was constructed in OpenSim environment and each student was given a 50m x 50m square land parcel with full ownership as the parcel owner. Students could modify their given lands to suite with their enclosure designs. Fig. 3.13 shows two successful student projects with creative ideas incorporated.
60
Figure 3.13: Completed student projects in OpenSim – Interactive Door Systems
The latest version available for the study was OpenSim v.0.6.9; it came with certain limitations due to the lack of support and partial implementations of the system components as it was being developed, making its behaviour less reliable. Most of these challenges were documented in the release notes of OpenSim, however some further unexpected behaviours occurred that affected student learning to a significant level. Land parcel access control was required similar to the work in SL. The same method of preventing public access was used, and it produced the virtual fences surrounding the parcels in the viewer applications indicating the word “No-Entry”. However, it failed to effectively stop students entering the parcels that they were not the owner of. Then, a workaround solution with extra effort was tried by adding all students but the parcel owner to the banned list of each parcel. It behaved reliably during the test runs, but in the actual student practical with multiple student accesses it failed to recognise the access control mechanisms reliably. Moreover, students could bypass the virtual fence and banned list protections by flying across the borders of the parcels. While none of the students were bothered by others having access to their lands in this case there will be learning activities in which parcel access restriction is essential.
With this set of HCI student projects (N=18), an effort to have a quantifiable measure about system usability was made. Student feedback about their experience with OpenSim was captured using a modified SUS in the context of OpenSim use. The standard SUS questionnaire has the word system as the subject of measure but for this analysis it was replaced by the word OpenSim to give more context for the student responses. Further, question items were modified to highlight the key areas of interest in examining OpenSim use for engaged learning, while keeping the basic model of SUS question items pairs. The given questionnaire items and the mean (µ) and standard deviation (σ) values for each statement are shown in Table 3.7. 15 students (83.33%) responded with completed questionnaires.
61
No Question µ σ
Q1 I think that I would like to use OpenSim frequently 2.00 0.93 Q2 I found OpenSim unnecessarily complex 3.27 1.22 Q3 I would like to use OpenSim for meetings with other students and staff 1.93 0.70 Q4 I think that I would need the support of a technical person to be able to
use OpenSim 2.33 1.11
Q5 I found the content in OpenSim interesting 2.40 1.12 Q6 I think that OpenSim would be unsuitable for assessed coursework 3.20 0.94 Q7 I would imagine that most people would learn to use OpenSim very
quickly 2.27 0.80
Q8 I found OpenSim disorientating 3.33 1.05 Q9 I felt very confident using OpenSim 2.73 1.03 Q10 I needed to learn a lot of things before I could get going with OpenSim 3.07 1.10
Table 3.7: Modified SUS Questionnaire items and relevant mean and std. dev.
Statements Q1 and Q3 were included to evaluate students’ willingness to use OpenSim. This helped us to decide whether the learning environment was attractive to students and whether they were willing to explore more usage in the future. The opposite statements were Q8 and Q6 respectively. Statement Q6 was of particular importance as it helped to evaluate the student perception of the specific area of the course assessment. Assessment methods have to be acceptable to students to be effective [181]. A virtual world assignment can be a failure if the assessment is not effective and convincing even though the rest of the course is successful. In order to inform the use of OpenSim with other courses, questions such as those outlined above would indicate how feasible that is. Statements Q5, Q7 and Q9 were included with the objective of assessing student difficulties with respect to using OpenSim. If the underlying learning environment is too complex students tend to learn more about the environment but not enough of the course content during their studies. Statements Q5, Q7 and Q9 were paired against statements Q2, Q4 and Q8 respectively.
The resulting usability value was 40.3, which indicated that improvements were needed in the system usability aspects of these OpenSim based learning activities. The research reported in this thesis shows how appropriate user guidance and training for lecturers and students can help achieve this. The responses for the questionnaire statements Q1 and Q3 indicate that students’ willingness to use OpenSim is below the average norm (3) of the scale. The opposite statements, Q8 and Q6 respectively, show above average values. As described previously, Statement Q6 was of particular importance since the response indicates that there should be some form of student confidence building of the course assessment. Responses to the statements Q5, Q7 and Q9 were below the average norm (3) indicating students were more towards to disagree with the statements. Even though the statements Q2, Q4 and Q8 are the counter statements respectively, only Q2 and Q8 showed agreeing response mean values. The statement Q4 had an anomalous mean value; the main reason for this was that the students had enough
62
technical knowledge to understand the system without extra help as they are CS honours students. However, for a non IT/CS student sample, extra technical assistance could be essential.
Discussions with the students also helped to see their view on using OpenSim as an educational medium. Apart from the instability of the OpenSim version, the students were quite happy to explore it further as a tool for their learning.
3.8 Concluding remarks
This chapter presented an overview of SL and OpenSim system models and the findings of user studies on student learning activities in these two MUVEs. Identified system models and complex functional behaviours of SL and OpenSim indicated a need for user support educating all types of users involved in using these environments for learning. Moreover, user studies indicated the significance of having a usable MUVE for educational activities. Not only do the complexities in these MUVE system behaviours give users a steep learning curve, but also introduce a challenging task to the academics to fit those into the institutionally established formal educational practices while preserving high usability. The next chapter presents the research findings and user support models for managed learning with SL and OpenSim.