Technology
After using the VP, respondents were asked a series of Likert questions concerning their satisfaction when using the VP and their view of the VP’s technical usability. Respondents also scored the VP‘s usefulness, their enjoyment using it, the difficulty in using it, their comfort using the technology, likeliness to use it again and if they would recommend it to others. Each question used a Likert scale of ‘’not at all’’ to ‘’extremely’’ (Figure 6.8)
For the questions regarding enjoyment, comfort, usefulness, likeliness to use the VP again and likeliness to recommend it to a colleague the median scores were ‘’quite’’ (IQR=1, except for questions on using the VP again or recommending it where IQR=2). Despite this, there were some differences in the spread of results across the Likert. Enjoyment using the VP had 73.4% of responses for ‘’moderately’’ or ‘’quite’’, likeliness to use the VP again had 53.1% of responses for ‘’moderately’’ or ‘’quite’’, and recommend to a colleague had 59.6% of responses for ‘’moderately’’ or ‘’quite’’. The
117
question regarding the usefulness of the VP was more positive as 80.9% of responses were for Likert scores of ‘’quite’’ and ‘’extremely’’.
For the question regarding difficulty using the VP the median score was ‘’not at all’’ (IQR=1), suggesting that the majority (86.2% for ‘’not at all’’ and ‘’slightly’’) did not find the technology difficult to use. The usability questions indicate that respondents, on average, found the VP usable and would it again and/or recommend it to others. A small group of individuals appeared to not enjoy using the VP and were more negative regarding its usability.
118
Figure 6.8 VP usability assessed via six questions using a scale of not at all to extremely
1.1% 1.1% 74.5% 0.0% 4.3% 2.1% 3.2% 0.0% 11.7% 9.6% 13.8% 7.4% 23.4% 6.4% 9.6% 9.6% 19.1% 18.1% 50.0% 47.9% 3.2% 50.0% 34.0% 41.5% 22.3% 44.7% 1.1% 30.9% 28.7% 30.9%
How enjoyable did you find using the virtual patient programme?
How comfortable were you using the virtual patient technology?
How difficult did you find the virtual patient programme to use?
How useful did you find the virtual patient programme?
How likely are you to use the programme again? How likely are you to recommend the programme to
a colleague?
VP Usability
119
Scenario content
There was a further group of questions that concerned the VP scenario and design. This included questions concerning feedback, VP questioning style and realism and were measured on a scale of ‘’strongly disagree’’ to ‘’strongly agree’’ (Figure 6.9). All of the questions in the series had a median score of ‘’agree’’ indicating that respondents on average were positive towards the VP. However, despite having the same median, the spread of data across the Likert did vary.
The MCQ options question was the question with the biggest range in responses, percentage score for responses of ‘’agree’’ or ‘’strongly agree’’ was 57.4% (IQR=1). Similar results were observed for the scenario playing out much like it might in practice (68.1%, IQR=1) and the likability of the feedback (73.4%, IQR=2). Conversely, the question asking if the VP had fulfilled its aim had 82.9% ‘’agree’’ or ‘’strongly agree’’ responses (IQR=0). The areas which had a greater spread across the Likert and a lower percentage of scores relating to ‘’agree’’ and ‘’strongly agree’’, namely the MCQs options, the scenario playing out like it might in practice and the likability of feedback, were
identified as areas to explore in the qualitative interviews (section 6.11.2).
Further questions asked whether respondents thought that the VP had changed their knowledge about NOAC counselling and their confidence, and whether this would result in a change in their practice. These questions also used Likert scales of ‘’strongly disagree’’ to ‘’strongly agree’’ (Figure
6.10). The spread of the responses for the three questions were similar and the median scores were
identical (median ‘’quite’’, IQR=0 for knowledge, IQR=1 for confidence and change of practice). The question regarding confidence had the lowest proportion of ‘’agree’’ and ‘’strongly agree’’ responses (70.2%) although this was close to the scores of the other two questions (72.3% knowledge, 75.5% change of practice). This suggests that users thought that the VP can change knowledge and confidence and could lead to changes in their practice. However, it is difficult to establish from this data whether there was a greater perceived effect of one of these outcomes over the other. Where there were responses that disagreed with this, they tended to be across all three questions when
120
Figure 6.9 VP application satisfaction measured across three Likert questions
0.0% 1.1% 2.1% 0.0% 1.1% 1.1% 2.1% 1.1% 10.6% 22.3% 17.0% 8.5% 6.4% 6.4% 9.6% 5.3% 12.8% 18.1% 23.4% 16.0% 11.7% 12.8% 14.9% 10.6% 62.8% 50.0% 46.8% 61.7% 56.4% 57.4% 44.7% 60.6% 13.8% 8.5% 10.6% 13.8% 24.5% 22.3% 28.7% 22.3%
The scenario was realistic to practice The scenario played out much like it might in practice The multiple choice responses met the needs of what I wanted to say to
the patient
The patient’s responses were realistic. The scenario helped me practice how to host a consultation on this topic The feedback provided at the end of the case was sufficient to guide my
learning
I liked the method of feedback at the end of the case when the patient verbally discusses my performance
The virtual patient programme fulfilled its aim of teaching how to counsel a new patient on rivaroxaban for atrial fibrillation
VP Application satisfaction
121
individuals did not like the VP. Some of the reasons for this are addressed by the free text questions within the questionnaire (section 0) and the issue was also explored in phase two of the study.
Figure 6.10 Implications of VP use across three Likert questions