MARCO LEGAL

dEvicEs

The huge variety of medical devices available calls for a flexible but integrated approach to accumulating evidence that a device is safe and will produce relevant benefits for health or health care. There are numerous study approaches available for building a portfolio of evidence for a device. One key difference between these approaches is whether the study will provide 1. direct evidence; 2. indirect quantitative evidence (e.g. short-term outcomes, intermediate/surrogate outcomes, accuracy measures) or 3. indirect qualitative evidence (e.g. results from different patient population or when the device tested differs from the device of interest). Whether a direct or indirect (linked) evidence approach is used, it is still possible to choose from a wide range of different study types or designs that can be used in each of these three research approaches. Each of these study types has its pros and cons (e.g. risks of bias).

In this appendix we describe traditional and novel study types and designs – everything from traditional randomised studies to novel, more tailored randomised stud-

ies, to semi-randomised and non-randomised designs – which can all be used in medical

device evaluations that are aimed at providing evidence of the benefits of a device (or its use) for health or health care. The advantages and drawbacks of each study type are given, and they are illustrated by examples of medical device evaluations taken from many different areas of medicine and many different types of devices. For background literature, see the references cited in the main text, which are complemented by some additional references (without being exhaustive) where deemed necessary.

A. Randomised (direct evidence) study approaches

Randomisation ensures that the groups being compared – one in which the device is used and one (or more) in which not the device but some alternative, compara- tive management (including placebo or sham management) is used – are created by random or chance splitting. Provided they are large enough, the groups are ‘the same’ 9  This section builds upon the extensive literature and books written on these topics, some of them referred to in the main text of chapter 4, and on the report Alternatieven voor Randomized Clinical Trials in onderzoek naar de effectiviteit van interventies. M. Langendam, L. Hooft, R. Scholten, P. Bossuyt. Dutch Cochrane Centre (DCC), Academisch Medisch Centrum, Amsterdam, februari 2013.

except for the device under study. All other modifying factors – the ‘confounders’ – are equally distributed across the comparison groups. Accordingly, any observed differences in benefits (and risks) between the groups can be assigned with a larger measure of likelihood to the actual difference in management, and thus to the device use; all other modifying (confounding) factors are ruled out due to the randomisation. Provided it is successful, randomisation allows researchers to draw conclusions about true cause-and-effect relationships between the device use and the health or other outcomes.

A randomised clinical trial (RCT) can have varying sample sizes, ranging from small trials to very large (mega) trials..10 _{Randomisation can be executed on the individ-}

ual or patient level, where the usual aim is to compare the effects of device use on patient-related endpoints, intermediate or long term. It can also be executed at the practitioner or end-user level, where the endpoints are user-relevant, such as a differ- ence in ease of use, efficiency, time-to-diagnosis, etc.

Device companies often struggle to fund and organise such trials. Randomised designs, especially the more novel and tailored approaches (see below under B), still can play a role by providing high-level evidence (low risk of bias) for the comparative safety, performance, benefits or effects, and even cost-effectiveness of devices. A1. Pragmatic or comparative effectiveness randomised trials

The best randomised design for direct evaluation of a medical device’s benefits or added benefits is the parallel randomised pragmatic design. This applies both to therapeutic and test devices, i.e. diagnostic, monitoring, screening or prognostic tests (see Chapter 4.3). Targeted individuals or clusters of individuals are allocated randomly, concurrently and in parallel to either the new intervention or the compar- ative (control) strategy. A large-scale, long-term pragmatic or comparative effec- tiveness randomised trial compares the use of the device in question directly with the best alternative care in the right population, measuring all relevant outcomes over the long term and with use as it would be in everyday practice. Of course in the case of diagnostic, screening or monitoring devices, the therapeutic actions dictated by the results of such devices are also part of the intervention.

Such a design would produce the most direct and valid evidence of whether the device will indeed produce the intended relevant health benefits, at an acceptable level of safety, as compared to prevailing care. Even the cost-effectiveness of the device use can be addressed. This is true both for devices that interfere with bodily systems in order to treat or alleviate specific health conditions (therapeutic devices) and for test devices such as diagnostic, prognostic, monitoring or screening tests that rather generate information based on which therapeutic actions are adminis- tered (see Chapter 4.3, main text).

10  Charlton B.G., Mega-trials: methodological issues and clinical implications. J R Coll Physi- cians Lond 1995;29:96-100.

Pragmatic randomised trials are a specific and – for device evaluations – argua- bly the best form of a parallel RCT. The key feature of the pragmatic trial is that the comparison is not a placebo intervention (see below) but an alternative intervention, usually best current practice, with no restrictions on their application. Pragmatic trials are designed to evaluate the effectiveness of the index intervention (e.g. device use) under routine practice conditions, as opposed to placebo controlled trials, which test whether a specific intervention-aspect (in drugs this is referred to as the ‘pharmaco- logical agent or substance’) is effective under optimal ‘experimental’ conditions. The pragmatic aspects of a trial may include, among others:

• the use of broad eligibility criteria to specifically select participants with heteroge- neous characteristics, conform daily care;

• having flexible strategies for the use of the device;

• include a variety of practicioners with different expertise regarding the device use; • include a variety of clinical settings;

• assess a variety of clinically meaningful or patient relevant outcomes.

Accordingly, the outcomes of pragmatic randomised trials are considered to have greater relevance for clinical practice and health policy makers.

The main disadvantage of pragmatic trials is that the participants and the practi- tioners are commonly not blinded, potentially obscuring the net effects of the index device. Their advantage, however, is that results are often directly applicable to daily care. The benefits (and risks) found in such trials are highly generalizable because pragmatic trials usually include participants who will be dealing with the intervention in the real world, recruit patients from a broader range of study sites (not only aca- demic or highly experienced medical centres and professionals), and have outcomes that often represent a full range (not only short-term or intermediate outcomes) of relevant patient and professional health outcomes.

A2. Randomised placebo-controlled trials

A placebo or sham-controlled parallel randomised design is the most traditional randomised design, and comes from the pharmaceutical domain. The major advantage of this design is that all possible influences from any source – patient and professional interpretation influences and other confounding factor influences – are controlled for due to the double-blinded, placebo-controlled randomised character of the study. This design is much less straightforward for the evaluation of medical devices than it is for medicines, mainly because devices are part of complex interventions where, for exam- ple, double blinding, and thus a full placebo or sham-management control group can be extremely difficult. Moreover, traditional randomised placebo-controlled studies are often conducted in an environment staffed by highly skilled and trained operators and by high-volume medical centres, with very specific patient inclusion and exclusion criteria. As a result, these trials deliver evidence in an environment that can be quite different from real-world, pragmatic use.

In the medical device sector, the comparative strategy often involves alternative, prevailing management or even no management (wait-and-see), see under A1. Sham or placebo-device interventions as comparison exist but are rare in device evaluations. When they do take place, they are almost always for therapeutic devices, and not for diagnostic, screening or monitoring devices.

ExamplEs of pragmatic trials usEd for mEdical dEvicE