ASUNTOS DE CARÁCTER ORDINARIO:

Three domains are included in this axis: 1) mobile CME content development; 2) mobile learner considerations and 3) faculty development.

Axis 3: Mobile CME content

Domain 5: Mobile CME content development 5.1 mCME pedagogy

5.2 mCME tools 5.3 mCME content Domain 6: Mobile learner 6.1 Target audience 6.2 Clinician preferences Domain 7: Faculty development 7.1 Capacity building

Domain 5: Mobile CME content development

This domain refers to the various approaches that can be used to deliver educational content through mobile tools. Three sub-themes are included: 1) mCME pedagogy; 2) mCME tools and 3) mCME content.

Sub-Domain 5.1 mCME pedagogy

Using mobile technology for learning has benefits similar to eLearning; namely, it allows the student to learn in his/her own location, while still being connected to faculty, other learners, and learning resources. Mobile learning, however, allows access to students who might not have a desktop computer or laptop, this facilitating learning ‘on the go’.

It is important to note that 20% (n=6) of global experts and one Vietnamese expert claimed that mobile technology “won’t fix everything” and that mobile learning should not replace in-person training or traditional skills related training. Granting the validity of this view, it is still important to understand how mobile technologies can be used. As one expert mentioned, “mobile technology adoption is increasing and it will be inevitable that mobile devices will be incorporated into routine medical education in the future”.

Nearly two-thirds (63% (n=19)) of global experts and more than one-third (36% (n=8)) of Vietnamese experts recommended using a blended learning approach when designing CME courses that are mLearning based. As mentioned earlier, blended learning can be defined as: combining in-person classroom and online learning in order to achieve

learning objectives. There are many examples in the literature about the various ways educators can combine virtual and in-person training, from having students complete some assignments or tasks online and discussing them during in person trainings to using mobile technologies to quiz or poll students during an in-person training. Two

Vietnamese experts were adamant that mobile learning should target knowledge and might not be good for skills building.

Learning via digital devices allows a wealth of data to be collected including information on learner performance, learning outcomes, and providing educators with information about when learner’s access content or how long they take to respond to a learning task. This learning outcome and “meta-data” can then be used to adapt the way the content is delivered and tailor it to individual learner’s needs. Among those interviewed, 40% of global experts (n=12) suggested that data collected through mobile and eLearning solutions can be used for “adaptive learning”, such that the technology can adapt to learner performance and target content delivery to those areas in which the learner is weak. Using adaptive learning techniques can have a strong impact on addressing individual competencies, ultimately influencing knowledge retention and performance.

Peer-to-peer and case-based teaching via mobile devices is emerging as an effective way to support knowledge and skill transfer among health professionals. Among all

interviewed, 17% reported (n=9) that mobile technologies can help facilitate better case- based or peer-to-peer learning. Examples of peer-to-peer learning include connecting the

learning directly to a clinician’s actual practice. This could be achieved by introducing virtual grand rounds whereby pictures of cases can be taken on phones and shared among a group of clinicians to review and comment on via mobile technology applications, with CME credits issued for the number of cases on which a clinician consults. Another example mentioned was using a mobile app to link a group of learners who all work on the same assignment together, thus promoting dialogue and discussion among clinician peers as well as more senior clinicians.

Virtual simulations have also been shown to have a positive impact on skills and knowledge acquisition. Ten percent of global experts (n=3) interviewed suggested that the future of mobile learning will involve more virtual simulations mimicking clinician- participant encounters. The use of games is also being used increasingly for medical education. Nearly one-third (30%) of global experts (n=9) highlighted the value of considering game theory and the entertainment associated with learning through a game. This could be a particularly important strategy for Vietnam as there is a fast growing culture of gaming in the country.

Another important consideration mentioned by interviewees is the need for a high degree of interactivity when designing mobile solutions to maintain learner interest and

connections to educators. Nearly one-third (30%) of global experts and 9% of Vietnamese experts mentioned that ensuring interactivity in mLearning is critical to engage learners for long-term knowledge retention. Another important factor for learner

motivation is ensuring that the content is relevant and interesting to the user. Of all interviewed, 17% of global and 41% of Vietnamese experts reported that motivating clinicians via mobile devices would require providing a choice of CME topics deemed relevant to clinician interests and work. For example, a physician in Hanoi might have very different learning needs as a specialist than a family medicine doctor practicing in rural areas. Having CME content and course options that are tailored to learners’ needs, rather than being driven by donors or implementing partners was described by

Vietnamese stakeholders as critical. Three Vietnamese experts also stated that mobile learning can save time and resources for the learner and educator as compared to in- person training. Experts also claimed that traditional eLearning approaches could integrate a mobile technology component as well to extend the learning environment.

Sub-Domain 5.2 mLearning Tools

In order to deliver a full CME course through a mobile device, educators need access to software platforms that allow them to build course content and facilitate mobile learning courses. As mentioned in chapter four, there are many open source and proprietary eLearning platforms with mobile applications that can deliver full course content. In Vietnam, the national eLearning network has chosen to use Moodle, while HAIVN is using the Zoom platform with their work.

Several global experts interviewed mentioned OppiaMobile, developed by Digital Campus, Inc., as a leading open source mobile application that is already integrated with

Moodle. The OppiaMobile application allows educators to build course content in Moodle and deliver the course either on the web or through a mobile application. OppiaMobile is an Android-based application allows learners to access educational content, activities, videos, text content, quizzes, and other features while in network or offline. Text-to-speech functionality allows for activities to be read aloud for lower literate populations. It also has a feature that allows automatic notifications of new courses added to a learner’s library, with badges and points can be earned for completing activities. Because OppiaMobile is integrated with Moodle, there is no need for

programmers or other IT specialists to be involved in maintaining or updating learning content. Any updates to course content can be pushed automatically to the learner’s device when they are in network. Tutors and other educators can access student activity to see course completion, the selection of videos or materials viewed, etc.

As mentioned in chapter five, other mobile technologies such as SMS, social media, and VoIP can also be used to deliver educational content, but each have benefits and

tradeoffs. In terms of using SMS to deliver educational content, 23% (n=5) of

Vietnamese interviewees mentioned SMS character length limitations and suggested that SMS be used mainly for knowledge-related education and to provide links to other technologies and in-person trainings. Vietnamese interviewees also mentioned that many private sector companies use SMS for advertisements and other commercial purposes. Therefore, delivering health content via SMS might be confused as spam and health workers might disregard messages. Additionally, global experts pointed to the fact that

SMS has been shown to be an effective way to amplify an existing training, but suggested that a high degree of interactivity is essential and exploring how SMS can link learners to a more complete set of learning resources virtually or in-person is important to achieve learning objectives.

Sub-Domain 5.3 mCME Content

The advances made in designing eLearning courses in Vietnam over the last five years are not insignificant. Over 300 eLearning courses have been developed, but are not yet fully implemented in the country. The MOH HIV/AIDS office also has a large repository of HIV related digital resources that still need to be structured into eLearning courses. Additionally, many universities and professional associations have repositories of content that could be adapted for delivery via mobile devices. The MOH plans to use Moodle to develop future courses. The MOH also plans to develop a database of all CME courses that can be stored and accessible at the national level. Given these two priorities, selecting Moodle, iHRIS (mentioned earlier), and a mobile application that can be integrated with these two platforms can help meet the needs of the MOH. Cataloging local CME content developed for in-person distance-based learning can support selecting key core courses that can be adapted for mobile delivery. Additionally, it will be

important to consider traditional medicine-focused CME courses as they are officially sanctioned by the MOH and traditional medicine services are offered in nearly all commune health centers in Vietnam.

The legislature in Vietnam allows for international CME courses or content to be used for official CME credit in country. However, the course or content still must be reviewed and approved locally. While there is a wealth of globally available open source health content available, stakeholders highlighted the need to ensure content is translated into

Vietnamese to ensure all learners can access and digest the information.

While it is important for educators in the country to be trained on designing eLearning and mobile courses using evidence based methodologies, 40% of global experts and 9% of local experts recommended considering using globally available, free, and open source medical education content and CME courses. ORB is an online content platform that globally crowd sources health worker training content and vets the content through an expert medical review for clinical accuracy before publishing it online (mPowering Front Line Workers 2015).

Domain 6: Mobile Learner

This domain refers to key considerations related to mobile learners themselves, particularly on how clinicians in country prefer to use mobile for CME.

Sub-Domain 6.1 Target Audience

Using mobile tools for learning is not for everyone. That is, it is not a “‘one size fits all” solution for every learner in need of CME credits in Vietnam. Moreover, not everyone is eager to change his/her ways of learning. Of those interviewed, 18% of Vietnamese stakeholders explained that there might be some general hesitation towards mobile learning as workers might be resistant to changing their learning behaviors to a digitally- based system. Additionally, delivering an entire course on a mobile device requires a smartphone to display videos and use advanced learning features. With the rise of smartphone ownership in the country, delivering a full curriculum for health professionals to view on their own device is feasible, but offering multiple learning opportunities for accessing CME will be important.

Addressing learner motivations was mentioned by 33% of global and 18% of Vietnamese stakeholders as critical to spur clinicians to adopt mobile learning for CME. Several types of motivations were mentioned during interviews. Ensuring that topics are relevant to learners’ needs was mentioned by 17% of global and 41% of Vietnamese stakeholders. Additionally, 17% of global experts and 23% of Vietnamese experts stated that official approval of mobile learning CME would motivate learners to access such CME content.

One global expert suggested that combining educational and entertainment content has been shown to improve learner adoption. Since CME is a requirement for licensure now in Vietnam, this puts regulatory pressure on the health worker to complete credits. Two global experts suggested revising job descriptions to tie participation in CME to

performance appraisals and job promotion.

Having content available on a mobile device allows flexibility for the health worker to learn anywhere, at any time. Ten percent of global experts and 18% of local experts noted that the expenses of traveling, accommodation, and food to participate in in-person training is a barrier to rolling out in-person CME widely throughout the country.

Participants in the FGDs supported this claim. Interviewees described mobile learning as a good option to reach clinicians with a mechanism to earn credits on their own time and at their own pace. However, one global and one local interviewee did note that health workers can often change phone numbers or SIM cards, which could pose a problem for reaching learners and monitoring their use of mobile content.

Understanding learner preferences by worker cadre, age, and location can provide insight into which groups are best to target with mobile learning solutions. Twenty percent of global and 27% of Vietnamese interviewees stated that mobile learning options might be more attractive to young, lower level cadres who might not have access to computers. Three Vietnamese interviewees mentioned that targeting rural based clinicians with

mobile learning options will help to overcome distance and travel barriers associated with attending in-person CME courses.

Sub-Domain 6.2 Clinician preferences

Each cadre of health worker has a variety of learning needs. They also vary by income level, with nurses, midwives, and physician assistants earning less than doctors. These variations can impact the types of devices that clinicians own, how much money they spend on data and voice plans, and their willingness to use their own mobile device for learning. The preferences related to each cadre of health worker are presented below.

Physician Preferences

None of the physicians in the FGDs had ever taken an eLearning course, but all had heard of such courses. Additionally, none had ever used mobiles for learning, though, the physicians revealed interest in mobile learning and emphasized that mobile devices would be more convenient if wi-fi were available. During FGDs, physicians mentioned they might be willing to pay for mobile CME access if fees were modest (McNabb Interviews 2015). However, all cadres noted that using mobile learning approaches was best suited for knowledge transfer and that in-service training should focus more on acquiring clinical skills (McNabb Interviews 2015). They indicated that mLearning topics should be more general and focus on preventative medicine, since their specialties require more focused technical content (McNabb Interviews 2015). Additionally, they felt that

ensuring interaction and connections with teachers and peers would be essential for effective mobile learning (McNabb Interviews 2015).

All physicians felt that mobile learning should not replace traditional in-person trainings, particularly for clinical skills training, and that mobile learning might be better suited for younger physicians (McNabb Interviews 2015). However, using SMS to send updates or knowledge focused CME would be acceptable and welcomed. Participants also noted the usefulness of using social media or linking mobile learning to eLearning courses

(McNabb Interviews 2015).

Nurse Preferences

Similar to the physicians, some nurses had heard of eLearning but none of them had ever completed an eLearning course, as many do not have personal computers or Internet connections at home (McNabb Interviews 2015). Most nurses reported being interested in mobile learning and suggested topics should be general, encompass preventative

medicine, be knowledge focused and be relevant to their learning needs. Overall, nurses, similar to physicians, suggested the need for more interactivity with mobile learning and noted that incorporating more videos and other media would make mobile learning more interesting (McNabb Interviews 2015). Like doctors, nurses claimed that older workers might not be as keen to learn via mobile devices. Additionally, not all nurses had phones or smartphones, so a variety of CME options should be considered for this cadre

appropriate target audiences for mobile learning, nurses generally preferred distance- based learning due to its greater convenience – they would not have to leave their homes and would have greater flexibility to learn on their own time (McNabb Interviews 2015).

Physician Assistant Preferences

Physician assistants who participated in the FGDs were all part of the mCME RCT research study described above. Overall they liked receiving information via SMS and usually worked together to find the correct answers. Additionally, they liked the flexibility of the timing to learn on mobile because they were usually tired at night and did not have access to Internet or computers at home (McNabb Interviews 2015). Similar to other cadres, they all agreed that mobile learning might be better suited for knowledge focused learning objectives and should be coupled with in-person trainings for clinical skills development. All preferred more interactive mobile learning approaches and felt it might not be a good approach for older learners (McNabb Interviews 2015). They cautioned about implementing mobile learning in mountainous areas that lack strong 3G coverage (McNabb Interviews 2015).

Domain 7: Faculty Development

This domain covers the needs related to building local faculty capacity to design and deliver mobile learning solutions. In Vietnam, work has been done to build the capacity of faculty to design and deliver eLearning courses. Interviewees emphasized the general lack of eLearning faculty capacity, with 32% of Vietnam and 27% of global experts

recommending further efforts to train faculty to design and deliver eLearning courses. Additionally 50% of Vietnamese experts claimed that faculty lack interest and time, as well as the IT knowledge to learn new online and mobile teaching methodologies.

However, many experts mentioned that younger faculty are often more keen to adopt new teaching approaches. Incorporating some specific considerations within those initiatives can help educators in Vietnam begin to incorporate mobile tools.

Some universities have experience offering more interactive eLearning courses in Vietnam that also use media such as videos or blogs. Seventeen percent of global interviewees mentioned the importance of using different audio/visual teaching tools, including instructional videos for skills and knowledge building. However, developing locally appropriate videos and content remains a challenge in academic settings. When asked about developing eLearning courses that follow evidence based best practices, one university interviewee noted that while the faculty had been trained to develop

educational videos, a notable challenge was that video equipment was outdated and hence inadequate; some instructors were also shy to record videos.

Lack of remuneration for faculty time spent developing CME, eLearning, or mLearning content was mentioned as a barrier to developing high quality CME courses. Two

Vietnamese academics mentioned it was a challenge to design eLearning courses in their universities because of the amount of time it takes to learn eLearning pedagogy and to develop courses. A lack of budget at the university and national levels to pay faculty was