El Trabajo Social en Neurología:

As it was already mentioned in this chapter, a number of technologies were used in this project to implement the prototype. The main focus was on utilizing the new features offered by the HTML5 standard and reusing existing JavaScript libraries (described in Chapter 6). These technologies showed to be a good choice as they allowed for rapid prototype interactions and easy code maintenance and execution.

The benefit of using HTML5 for extracting video/audio fragments and drawing concept maps was that it did not require any additional plugins to be installed on the user side. This can be considered an advantage over Flash-based implementations which depend on plugin and sometimes may require a specific version of this proprietary plugin to perform correctly. Using HTML5 helped to ensure that the prototype features would work on most web browsers as expected.

Along with the benefits, these technologies also created a number of challenges in development. Among these was a relative novelty of the HTML5 standard compared to the established and widely-used HTML4. The Web Hypertext Application Technology Working Group23 (WHATWG) and World Wide Web Consortium24 (W3C) started working together on HTML5 standard in 2007 (World Wide Web Consortium, 2012). However, at the time of the prototype development in 2011 this standard was still a working draft. As a result, some browsers did not provide full support for the new features offered by HTML5 such as embedded video/audio and canvas elements. To provide dynamic user-website interactions and enhanced user interfaces, the imple- mentations also used scripts, libraries, and modules based on JavaScript. This can be

23_{http://www.whatwg.org(Accessed April 16, 2012)} 24_{http://www.w3.org(Accessed April 16, 2012)}

a potential problem for users who have JavaScript disabled on their browsers as they would not be able to use features that rely on JavaScript functionality. Especially, it can be a case for institutional environments where computers are usually set up by the IT support groups. According to the researcher’s experience, these tend to disable JavaScript executions for security reasons (for example, to prevent cross site scripting attacks).

Despite these challenges, it can be concluded that the technologies employed in the prototype development offered a wide range of libraries, modules, and APIs that made implementations less complicated and time consuming. At the same time, these tech- nologies helped to develop high quality features and functionalities, as well as acceptable user interfaces.

6.6

Summary

This chapter presented design and implementation of additions and improvements of the ePortfolio system based on the needs of the stakeholders and lifelong learning success guidelines discovered in the literature. A set of features was introduced to the stan- dard ePortfolio installation with the aim to address various aspects of lifelong learning support in universities. These features are expected to support understanding, develop- ment an showcasing of lifelong learning skills, learning progress tracking, management of ePortfolio content and knowledge, communication between students and lecturers, and better control over access to the ePortfolio resources.

From this stage, to understand whether new implementations meet the needs of the stakeholders, a formal evaluation has to be undertaken. The next chapter looks at the improved ePortfolio system prototype from the perspective of lecturers and students with various levels of experience.

Evaluation

In the course of this thesis the requirements, design and prototype implementation of the ePortfolio system that can support lifelong learning has been discussed. This chapter focuses on the evaluation of this research and its contributions based on three studies. The studies were designed to look into the specific aspects of lifelong learning support and understand how well developed features satisfy the requirements identified earlier by the lecturers and students.

The number of studies represents perspectives of all of the stakeholders that have been involved in this research earlier, i.e., lecturers and students. Reason for this was to ensure that all of the perspectives are being captured asdifferent stakeholders often have different perspectives (Hevner and Chatterjee, 2010, p. 111). In addition, the evaluation from the students’ perspective has been split into two studies as the requirements elicitation stage discovered that the perception of lifelong learning depends on the maturity of students. Therefore, it was decided to use different evaluation approaches for each group of students.

The first section of this chapter describes the approaches and data collection methods used in the evaluation stage. Further, each of the three studies is presented with the de- tailed participants profile description, exercise protocol followed by the study, artifacts collected and the results of data analysis. Each study section ends with conclusions that look into recommended improvements to the ePortfolio system prototype and its processes.

7.1

Design Overview

As this chapter aims to evaluate this research and its contributions, the relevant research question and its sub-questions are restated here:

How does the extended environment meet the needs of the stakeholders in university teaching and learning contexts?

• How can lecturers use new features to provide students with their guidance and help them to understand lifelong learning skills?

• How can students address lifelong learning skills using new features?

• How can new features help students track their learning progress, manage ePort- folio knowledge and organize content, demonstrate and share their achievements with others?

To answer these questions, three studies were carried out independently from each other. The results were used to evaluate the developed prototype from three different perspectives: lecturers, mature students and less experienced students. Each study followed its own exercise protocol described in the related sections. Detailed design of each study can be found further in this chapter in Sections 7.4, 7.5, and 7.6 respectively. Data collection for analysis was performed using both quantitative and qualitative methods to support the principles of multiple sources and multiple perspectives of data advocated by many researchers (Yin, 2009; Maimbo and Pervan, 2005; Marshall and Rossman, 2011). The following techniques were used over the course of all studies:

• Behaviour observations made by the researcher during all studies.

• Observation data in form of digital photographs taken during the group experi- ments.

• Audio recordings of the face-to-face interviews collected to facilitate more thor- ough interview analysis.

• Open-ended questions asking for participants opinion on the features and tools used in the studies.

• Close-ended questions that required participants’ evaluation based on ten point scale, from not useful at all tohighly useful, included in exit questionnaire.

• Physical artifacts in form of paper records made by participants of the group experiment.

• Digital artifacts in form of electronic records in the ePortfolio system made by participants of the group experiment and case studies.

• System logs demonstrating ePortfolio system use by case study participants.

• Additional evidence, comments and opinions collected by the researcher through informal discussions that would help to support analysis and conclusions.

Audio recordings as well as digital photographs were collected with the permission of the participants and performed without interrupting the process of each study. More detailed description of data collection methods for each study can be found in the relevant sections this chapter and associated appendices.

7.2

Evaluation Limitations

It is important to remember that in the context of this thesis, it is impossible to fully evaluate lifelong learning. Ideal evaluation would include providing students with a technical support solution developed in the course of this project and monitoring their activity through the years of studying at the university. However, this approach appears to be not feasible due to the nature and scale of this research project, its time and resources constraints. Even if it had been possible to involve student participants for a longer term, the issue would have been for them working in a system they cannot have access to long-term as the Mahara ePortfolio system used in this project was different from the Mahara installation recommended by the university.

Overall, no existing evaluation design has been found that would suggest an acceptable approach. Therefore, the evaluation stage in this project followed its own design that attempted to explore and analyse the implementations from the perspective of the lifelong learning stakeholders.

7.3

Ethical Considerations

Similarly to the earlier stage of the requirements elicitation, the Massey University Ethics Approval process was followed for the evaluation studies of this research. Anal- ysis of the evaluation design with the Human Ethics Chair concluded that none of the three studies required full ethical approval. Therefore, a “Low Risk Notification” was submitted to the Massey University’s Low Risk Database.

Ethics documentation, such as the Ethics Approval Letter, student information sheets and participant consent forms, used for this research project stage, can be found in Appendix E.

7.4

Study One. Exploratory Evaluation by Lecturers