de 101 3.Servicios educativos de calidad a alumnos de nivel Profesional

Educational tools can be assessed by improvement in standardised testing or other variables such as user learner satisfaction/ acceptability of the material. User evaluation is a constructive and valuable assessment method to determine the

educational success of multimedia [51]. In addition users’ feedback is likely to improve the effectiveness of multimedia tools. Therefore an important component of this study was to evaluate the acceptability of the educational tools.

Expert evaluation: There was a low response rate (21%) to request for evaluation of the tool by surgeons and educational experts. Busy clinical or educational schedules are the most likely reason to account for the poor response rate.

172

Although the interface design for both tools were scored positively, the open tool scored higher with regards to all aspects compared to the laparoscopic tool,

particularly with regards ease-of-use to navigation. This may have been related to the increased complexity and interactivity in the laparoscopic tool.

All statements related to learning process and features received positive responses; although 30% were unsure regarding ‘accommodation of learners' individual

differences’. The reasons for this are unclear and warrant further clarification.

Experts’ responses suggest multimedia learning tools can be more effective compared to other traditional adjunctive educational platforms, in particular surgical textbooks. If multimedia tools are deemed to be more effective than study days/ courses, this can potentially have cost saving implications for trainees.

Experts felt the educational content was applicable for ST3-8 level trainees, and this was an important factor considered during development. Dividing the operation into steps and subtasks allows trainees to self-direct to relevant educational material. Whilst being able to improve trainees’ cognitive skills, experts felt the most appropriate time to utilise the tools was prior to operating lists.

Trainee evaluation: The high response rate (81%) of the multimedia group to the online evaluation tool suggests trainees were motivated by the technology.

The results highlight the success of the interface design following implementation of multimedia design principles. Although most aspects of interface design (ease of use, navigation, and mapping) were well accepted, users approved the open design more strongly (table 4.9). The consistent structure of the open interface design may be a factor. Screen design (focussing on animation, text, video, voiceover, and graphics) and the integration of all these types of media onto both tools were well accepted.

Users responded positively to learning process and features (table 4.10). The majority of responses indicated that both tools achieved goal orientation by focussing on cognitive skills acquisition. Multimedia allows trainees to take the initiative and responsibility for their own learning, but this requires self-motivation [54]. Users

173

generally felt the tools created an interactive learning experience and were intrinsically motivating which is an important factor in self-directed learning and adult learning principles.

Multimedia allow users the opportunity to search or navigate to educational media of their choice; multimedia also has the added advantage of repetition of media. Users can become more engaged with the content as they have control over the pace and the sequence of the content [54]. The majority of trainees (79%) agreed positively that the tools gave unrestricted user control. A small proportion disagreed and a number of trainees were unsure, suggesting further development of the tools is required to optimise this aspect.

Accommodation of a range of learner differences enables different learners to

organize the information in a manner that reflects their learning style. Results indicates the tools are applicable to all trainees and suggest that multimedia adapts well to individual differences due to the high degree of learner control and the ability to cater for varying learning styles [150]. This also points to balanced use of “scaffolding” [151] having been successfully implemented.

In both tools efforts were made to engage users to focus on media for learning and not to divert attention to unnecessary information; in this way learning demands on the user could be minimised. Increased interactivity may have accounted for user disorientation, particularly in the laparoscopic tool, resulting in some negative responses. However, in general, strongly positive responses to learning demands placed on the user suggest that ‘cognitive load has been minimised’ this points to adherence to multimedia design principles during multimedia construction.

Both tools compared favourably with current teaching methods suggesting multimedia has as a role in surgical education. Though the interface and learning features are important, multimedia’s capacity to also allow flexibility in learning and self-paced learning are additional positive factors. This adheres to adult learning principles and is an important factor in modern surgical training. Users agreed the lack of human interactivity is a drawback, suggesting they still value personal communication and feedback from expert instructors. Implementing self-assessment and feedback into the

174

multimedia tool, similar to the CEVL modules [142], will improve the aspect of feedback.

Trainees, in general, felt that the multimedia had improved their cognitive surgical skills. This may reflect the higher scores achieved by the multimedia group in the assessment tests. While the majority felt that their factual/ anatomical knowledge and anatomical place recognition skills had been enhanced, a proportion of users (25%) disagreed/ strongly disagreed that multimedia had improved their decision making skills. The reasons for this are unclear; we can speculate that these trainees may feel that multimedia is more applicable for attainment of procedural knowledge and that development of decision making skills should take place in the intra-operative environment. These results warrant further investigation. Ultimately the ability for multimedia to improve surgical skills needs to evaluated in clinical situations (i.e. studies on intra-operative decision making), and not merely by subjective evaluation of skills improvement.

Although there are areas that require further development and improvement, the responses to the learning process and features, and training tool appraisal statements were all generally positive from both experts and trainees. Favourable responses should be treated with some caution, as they may represent the enthusiasm of a small, self-selected group of motivated trainees and sympathetic experts, the latter being a selected group known personally to the research team. This would have introduced a certain amount of inherent bias and needs to be taken into account.

Overall, multimedia is a well received and valued adjunctive educational tool by both experts and trainees. Evidence from the literature and trainee responses’ suggests feedback is a crucial component in the evaluation of multimedia educational tools and needs to be integrated into the design.

175