A structured data extraction form was used to guide the review of the submission by Carl Zeiss, UK, to NICE (see Appendix 4). The MS evaluated the cost-effectiveness of INTRABEAM in early breast cancer patients when compared with radiotherapy, which is usually given in the UK over 3–6 weeks as WB-EBRT. The total costs, QALYs gained and cost-effectiveness associated with the intervention and comparator under consideration in the appraisal were reported in the MS. The perspective adopted in the submission was that of the NHS, capturing direct costs and benefits only. A systematic review of any relevant
cost-effectiveness models was not conducted. Very limited information on the model was presented in the main submission document and, although further details were contained within the Microsoft Excel (Microsoft Corporation, Redmond, WA, USA) model, these too were limited.
Modelling approach
A multistate Markov model, developed in Microsoft Excel, was used in the submission. The model used a cohort of breast cancer patients aged≥ 55 years who were disease free after WLE. The economic model was based on the results of the pre-pathology stratum of the TARGIT-A trial65with 2298 patients. This was
because results were less favourable in post-pathology stratum (see Chapter 4, Assessment of effectiveness) and the submission recommended that INTRABEAM be used in pre-pathology patients only (MS, pp. 3–4). It was not reported whether the model was constructed de novo or adapted from another previously existing model. The model consisted of four health states:
l disease free
l local recurrence treated by mastectomy/lumpectomy
l non-breast cancer death
l breast cancer death.
Patients in the disease-free state could remain in that state or transition to either local recurrence or non-breast cancer death. Those in the local recurrence state could remain in that state or die from either non-breast cancer or breast cancer-related deaths. The two death states were the absorbing states. The analysis was conducted for a time period of 20 years with an annual cycle length.
Assumptions
The manufacturer’s model made the following assumptions:
l After local recurrence, INTRABEAM patients would undergo salvage lumpectomy.
l After local recurrence, WB-EBRT patients would undergo salvage mastectomy. There is also an undocumented assumption that all patients undergoing mastectomy have reconstruction, which is reflected in the high cost of mastectomy.
l The death rate in disease-free patients was equal to that in the general population.
l An average of 23 fractions of WB-EBRT per patient were delivered, based on 15–30 fractions in the clinical practice.
l All patients were given INTRABEAM concurrent with initial lumpectomy (pre-pathology stratum of TARGIT-A trial).
A few of the model assumptions are not relevant to UK practice. The model assumed that INTRABEAM patients would undergo salvage lumpectomy after local recurrence; however, clinical experts advised that in the UK most patients would undergo mastectomy after local recurrence instead. Furthermore, the undocumented assumption that all mastectomy patients would undergo reconstruction is not in line with UK practice, as only around 31% of the patients undergoing mastectomy will have reconstructions, as shown in the independent model discussed in Methods for economic analysis. In addition, the assumption of using an average of 23 fractions of WB-EBRT per patient was not appropriate as the current standard UK practice is 15 fractions.
Critical appraisal of model
The manufacturer’s economic evaluation was appraised for methodological quality and generalisability to the UK NHS using a checklist adapted from the NICE reference case requirements and the Philips et al.58
checklist (Table 20). The evaluation met half of the requirements for methodological quality and
generalisability, and the remaining criteria were either not met or unclear; therefore, the evaluation did not fully meet the NICE reference case. A brief description is presented below.
The manufacturer’s evaluation provided a clear statement of the decision problem to be addressed, which appeared to follow the scope for the appraisal issued by NICE. Although the comparator included
WB-EBRT, which is routinely used within the NHS, its appropriateness is questionable as the number of WB-EBRT fractions used in the UK practice is 15 compared with 23 fractions used in the model.
Six out of 33 centres in the TARGIT-A trial were based in the UK and centres were allowed to follow local policy for WB-EBRT delivery. The MS reported 23 fractions as the average of the range between 15 and 30 fractions being used in all the countries in the trial, but it was not clear if this was a weighted average of fractions used in the trial or a midpoint. The perspective adopted in the model was appropriate and, although the MS reported that the analysis was UK based, limited details were provided on the baseline characteristics of the patient population. A Markov modelling methodology was used, which seemed appropriate given the clinical nature of breast cancer; however, the AG considered that the reported model was a simplified structure with only four health states and that an additional health state for progressed disease would have been appropriate. Another limitation was that a lifetime horizon was not adopted.
Patients entering the model were aged 55 years (on average) and were followed for 20 years. This time span might not reflect the entire follow-up period of the disease. Patients transitioned through the health states in annual cycles, which is an appropriate assumption. The model structure was presented
diagrammatically but no justification of the key assumptions and description of the data inputs used was provided. Measures of clinical effectiveness were obtained from a single study;65however, no other
relevant trials were identified by the SHTAC’s systematic review. Benefits for the model were measured in QALYs using standard gamble for measuring utility, although the source study was dated 1997.135
It was not clear if a systematic review was conducted to identify the study. The model extrapolated local recurrence and survival data beyond 5 years by tacitly assuming an exponential fit to time to local recurrence; however, the AG considers that a log-normal distribution would be the best fit based on comparison with external data (see Data sources). All benefits and costs were discounted at 3.5% as outlined in NICE guidance. Uncertainty was assessed through PSA and no one-way or scenario analyses were conducted. Finally, no details around model validation were provided.
Estimation of effectiveness
Data on effectiveness for both the intervention (INTRABEAM) and the comparator (WB-EBRT) were derived from a single RCT (TARGIT-A) by Vaidya et al.65and 5-year cumulative risks reported in the source study
were converted to annual probabilities and populated in the model. It was not reported whether or not a systematic review was conducted to identify the source study; however, no other relevant trials were identified by the SHTAC’s systematic review (see Chapter 4, Quantity and quality of research available). No adverse events were included in the analysis, which was considered appropriate by the AG.
Estimation of quality-adjusted life-years
Health-related quality-of-life utility values were assigned to patients in the disease-free state, those undergoing salvage lumpectomy and those undergoing salvage mastectomy. A utility value of 0.92 was assigned to patients in the disease-free state, a value of 0.87 to patients undergoing salvage lumpectomy and a value of 0.82 to those undergoing salvage mastectomy. The MS obtained these values from a single study by Hayman et al. published in 1997.135No details were provided of the method of deriving
TABLE 20 Critical appraisal checklist of the manufacturer’s economic evaluation (based on Drummond et al.57
and Philips et al.58
)
Item number Item Carl Zeiss
1 Is there a clear statement of the decision problem? Yes
2 Is the comparator routinely used in UK NHS? ?a
3 Is the patient group in the study similar to those of interest in UK NHS? ?b
4 Is the health care system comparable to that in the UK? Yes
5 Is the setting comparable to the UK? Yes
6 Is the perspective of the model clearly stated? Yes
7 Is the study type appropriate? Yes
8 Is the modelling methodology appropriate? ? 9 Is the model structure described and does it reflect the disease process? Yes
10 Are assumptions about model structure listed and justified? No 11 Are the data inputs for the model described and justified? No
12 Is the effectiveness of the intervention established based on a systematic review? No 13 Are health benefits measured in QALYs gained? Yes
14 Are health benefits measured using a standardised and validated generic instrument? Yes 15 Are the resource costs described and justified? No
16 Have the costs and outcomes been discounted? Yes
17 Has uncertainty been assessed? ?
18 Has the model been validated? No
?, unclear.
a Different number of WB-EBRT fractions used in the model (23 fractions) than standard UK practice (15 fractions). b Baseline characteristics were not provided.
these values or the rationale used. The source study135used a standard gamble approach to estimate utility
values, which were not obtained from the general population. This is a limitation as it was shown in the systematic review of HRQoL (see Southampton Health Technology Assessments Centre’s systematic review of health-related quality-of-life studies) that there were several other more recent and relevant HRQoL studies that used the EQ-5D measure.
Estimation of costs
Treatment unit costs were obtained from the following sources: expert opinion, reference costs 2012–13,136
payments by results tariff 2013–14,137and the study by Wolowacz et al.138As with clinical effectiveness
and utilities, the methods of deriving the costs were not adequately described. The costs associated with travel/parking/accommodation were appropriately not included within the WB-EBRT arm (it was stated that these expenses might range from £50 to £100 per patient per fraction delivered).
The validity of the costs estimates is questionable. The cost of INTRABEAM per patient was obtained from expert opinion and although the manufacturer provided the cost compositions of INTRABEAM, it was not transparent in explaining the assumed cost per patient. In addition, cost of WB-EBRT was obtained from an inappropriate Healthcare Resource Group (HRG) code, the code used in the model for WB-EBRT was for ‘other radiotherapy treatment’, whereas the AG considers that the HRG code description required for the purpose of this analysis is‘deliver a fraction of radiotherapy on a megavoltage machine’, which includes WB-EBRT delivered by linear accelerator, as per the NICE scope. The AG considers that HRG codes SC22Z and SC23Z are required for treatment delivery, and SC45Z, SC46Z, SC47Z and SC48Z are required for WB-EBRT (see Data sources and Table 31). Costs were only varied by± 10% in PSA. There were also inconsistencies in the sources used to populate the reported costs; for instance, the costs of treating post-INTRABEAM local recurrence (salvage lumpectomy) and that of treating post-WB-EBRT local recurrence (salvage mastectomy) were obtained from payments by results tariff 2013–14, whereas the cost of
WB-EBRT was obtained from the reference costs 2012–13.137The use of reference costs is preferable and
would be considered standard practice.
Cost-effectiveness results
The base-case results from the submission are shown in Table 21 and indicate that INTRABEAM is associated with higher QALYs and lower costs. The submission states that the incremental analysis shows dominance of INTRABEAM over WB-EBRT.
One-way sensitivity analyses and scenario analyses were not conducted. A PSA was undertaken using Monte Carlo simulation with 1000 iterations. The cost parameters in the model were assigned to beta-project evaluation and review technique (PERT) distributions and beta distributions were assigned to utilities. For the cost parameters, the AG considers that gamma distribution would have been a more standard choice. It is not usual practice to assign beta-PERT distribution; however, it is expected that this would have little impact. For the PSA, at the £20,000 and £30,000 willingness-to-pay (WTP) thresholds, INTRABEAM has the highest probability of being cost-effective, at 100% for both thresholds.
TABLE 21 Base-case results for the Carl Zeiss submission
Intervention Mean QALYs Mean cost (£) ICER vs. WB-EBRT (cost/QALY)
INTRABEAM 13.230 14,461 Dominates
WB-EBRT 13.223 20,926
Critique of the manufacturer’s submission
l The MS provides very limited information on model structure, baseline characteristics of the patient population and setting.
l Limited information is provided with respect to input parameters such as costs and utilities. The MS is not transparent in providing the methodology adopted to inform the input parameters.
l The method to derive costs of INTRABEAM is not clear.
l No rationale is provided for using the specific distributions assigned to the parameters.
l The method of extrapolation of local recurrence and survival data is not justified.
l The number of fractions for the WB-EBRT arm used in the model (23 fractions) is higher than UK practice; this will lead to an overestimation of WB-EBRT costs.
l The manufacturer’s model assesses health benefit in terms of QALYs, which is a valid measure of health in the UK NHS setting. The source study135used standard gamble from a 1997 publication to
estimate utilities. No details were provided as to whether or not a systematic search was conducted to identify utilities for the model.
l Model validation was not conducted; hence, the generalisability of model results remains questionable.
l Probabilistic sensitivity analysis was conducted for only 1000 simulations and no one-way or scenario analyses were conducted. Limited sensitivity analyses conducted around the base-case model results raise questions on the robustness of the model predictions.
l In summary, results of the MS model should be viewed with caution owing to the methodological and reporting limitations outlined above.