The purpose of the pilot trial was to determine whether our telehealth implementation of outpatient consultations for the Department of Surgery at that hospital could be used to deliver those consultations.
Attitude of the clinicians
The clinicians approached this trial from the point of view of the welfare of their patients and they did this in five ways:
• They selected patients according to suitability criteria defined in the research protocol for this pilot trial.
• They paid great attention to ensuring that they would be able to map their clinical practice onto the resources of the telehealth system during the training session.
• They treated the telehealth phase of each consultation with full seriousness, taking as much time as they needed in order to reach a satisfactory conclusion.
• They avoided burdening the patients and family with the artificiality of a repeated face-to-face consultation.
• When aspects of the consultation moved out of the strict scope of the trial, as they did several times, the clinicians took the obvious and direct actions needed for the situation without concerning themselves with any rigid trial protocol.
This attitude contributed strongly to the success of the trial, in which the surgeons agreed that in 39 of the 44 cases the telehealth phase was satisfactory in its own right. A consequence of this positive response and of several other largely positive
quantitative responses to the exit questionnaires is that the researchers needed to pay close attention to the qualitative responses in the exit interviews with the clinicians, and to the observational data, in order to identify problem areas with the trial.
Attitude of the patients and their families
The patients were invited to take part in this trial based on the assessment of the their treatment progress by the surgeon, using criteria, specified in our proposal to the
hospital’s Human Research Ethics Committee, that would protect the patients’ and their families’ interests. Further, the patients at this hospital had generally formed a strong positive relationship with the hospital and its clinicians. The Royal Children’s Hospital has had iconic status for quality children’s care in Victoria for more than half a century. The patients and their families were generally very positive about being involved in this trial. One parent commented specifically that she was pleased to be able to repay the care that the hospital had shown to her and child by taking part in this trial.
The responses of the patients and families to the quantitative items in the exit interviews were also largely positive. For example, 35 out of 42 respondents answered “Yes” to the hypothetical question on whether they would have used such a system at a local healthcare facility for this consultation. Again, a consequence of this situation is that the researchers needed to pay close attention to the small number of negative responses and to explore the qualitative responses in the exit interview to uncover any criticisms of the trial.
Learning curves
A feature that I was not expecting was the learning curve that the clinicians traversed as they progressed through the pilot trial. In conventional telehealth I have not found reference to the need for the clinicians involved in trials to learn to use their telehealth systems, and this may be due to the less complex nature of the healthcare and the simpler nature of those telehealth systems. In our trial we observed, and we heard the clinicians themselves reflect on, the process that they went through to learn to use a telehealth approach to their work.
There were several aspects to this learning process:
• The surgeons, who were the senior clinicians in our trial, spent most of their training session mapping their existing outpatient approach onto the features supported by the telehealth system. This was relatively straightforward because we had designed the telehealth system to support what we had observed in several outpatient clinics. Even so, each surgeon at this level of specialisation had his own particular approach to his work, and each surgeon wanted to conduct his clinic differently to his colleagues.
• The clinic assistants, who themselves were highly skilled in their own specialty, had to learn a new role, which was to manage the interaction between surgeon and patient/family, and to conduct any examination of the patient in a way that the surgeon could fully appreciate.
• The surgeons and clinic assistants needed to develop a new professional relationship with each other in terms of working together to conduct these
outpatient clinics. These particular relationships are not present in the normal course of their work – the senior surgeon might review the results of a
consultation taken by the registrar, but they do not normally conduct outpatient consultations together.
• The surgeons were initially uncertain about the types of consultations that they could perform with this system. In the exit interview after their first clinic, two surgeons agreed that they would not be comfortable seeing a new patient or discharging an existing patient with a telehealth consultation. Four weeks into the trial, however, we saw them doing both of these and reaching an agreed endpoint in each consultation.
• For these particular surgeons, the range of health conditions for the patients was wide. Even towards the end of the trial the patients were presenting with clinical situations which were new for this trial. These cases typically required the surgeon and assistant to continue their learning process for the telehealth system. During the course of the four-week trial the surgeons and assistants repeatedly observed in their exit interviews that they were steadily learning to use the telehealth system, and in particular they were learning how to effectively work with their clinical partner (surgeon or assistant). This is an important point for future researchers in tertiary outpatient telehealth. Even though the specialists are highly competent in their own specialty, the complexity of conducting outpatient consultations in a telehealth mode requires them to learn how to do so, and the evaluation responses in the first few weeks of a trial are likely to reflect this learning process.
Role of laboratory studies
Pilot, or early stage trials are likely to follow a period of development for the telehealth system. The case studies in this thesis show that there can be a role for laboratory studies in this period of development. There will, of course, be technical unit testing that verifies the correct functioning of the various hardware and software components, but this will not be sufficient to ensure that the system supports the required user behaviour. User behaviour for this level of telehealth needs to be explored in the context of scenarios that match in some way the expected behaviour in the actual trials. The tension in this situation is that, while it is very important that the telehealth system support the required user behaviour at the very start of a trial with actual patients, the clinicians at this level of healthcare are likely to be too busy to take part in pre-trial evaluation studies. The three laboratory studies which are presented in this thesis illustrate an approach to resolving this tension. We can think of this approach in the style of a Pattern Language (Alexander, Ishikawa et al. 1977), where these studies identify the (design) problem, namely that it is impractical and inappropriate to ask highly trained clinicians to take part in studies of relatively low-level components of the developing system. These studies then illustrate a pattern for addressing this problem, which is to abstract sufficient elements of the actual scenarios to exercise the system’s support for the user behaviour, and to also abstract from the actual scenarios the criteria for evaluating the abstracted scenarios.
Pilot trials in the hospital setting
The pilot trial showed that it is very important to conduct early trials of telehealth systems for tertiary outpatient consultations in an actual hospital setting. To attempt a trial in a laboratory setting with actors playing the roles of the participants would not capture anything of the high-level clinical skill that we observed during the trial, and any evaluation would only reflect the quality of the initial briefing for the actors. To attempt a trial with actual patients in any place other than the hospital would also be very difficult. We observed many dependencies from within our trial on the wider hospital setting, including radiology, patient management, medical records and close proximity for the clinicians to their normal working locations. In our case we had both the surgeon’s and the patient’s rooms within the same hospital but, even if the trial were to span two healthcare facilities (a tertiary hospital and a regional healthcare facility, for example) this same wider context would need to be provided.
Emergence of previously hidden requirements
Even though we had one and a half years’ of interaction with the hospital during our development phase, when we arrived to install the telehealth system and train the clinicians we found that eleven of the twelve clinicians had only recently been involved in our project and had no familiarity with our telehealth system. During the four weeks of our trial two of our surgeons had prior commitments, and two other surgeons
substituted their own outpatient clinics on these occasions. Nine months after the trial, two of the main assistants in the trial had resigned from the hospital and the other assistants had moved to different hospitals as part of their long-term training.
This meant that for five of the six training sessions, and for five of the twelve clinics, we were working with clinicians in a “first time” role. In each case the clinicians identified additional requirements for the telehealth system, even though we were at a late stage of fine-tuning the development. Some of these requirements were preferences to which the clinicians could adapt, but some were mandatory for the success of their clinics.
At this level of healthcare, the specialists are likely to each have their own particular way of doing things. With the volatility of staff availability that we observed, it is likely that researchers on future pilot or early-stage trials will meet similar situations. We dealt with our situation in two ways:
• As an implementation approach, we designed a level of generality, redundancy and flexibility into our system so that we could make adjustments to its
functionality during the trial. We used this flexibility on two occasions, implementing an additional video camera to observe standing/walking patients and configuring the output of the hospital’s computer to show X-Ray data on the display tablets for both surgeon’s and patient’s rooms.
• As an evaluation approach, we treated these uncovered requirements as out-of- scope events which we noted as valuable input for any subsequent version of our telehealth system.