Working with Dementia UK
We were lucky to work in partnership with Dementia UK, both in planning the project and throughout all of its elements. Dementia UK wanted to evaluate the AN service, which is a large part of what it does, and to have this evaluation carried out by independent researchers. In this, it demonstrated a wish to learn from impartial evaluators about what Admiral Nurses currently do and to use this learning to improve what they do in the future. This openness to outside scrutiny is, perhaps, not as common among health and social care providers as it might be.
Officers at Dementia UK enabled access to their administrative database and answered very many questions from the research team to make it possible for the team to carry out the analyses on it. They also provided a vital link between the research team and the AN services selected for the survey, ensuring that paper questionnaires and electronic links to the questionnaire were distributed in accordance with the research team’s sampling strategy. A senior member of the Dementia UK management team was an ex officio member of our project advisory group in order to facilitate all of our links with the organisation, but did not have an advisory role in relation to the conduct of the research.
Despite all of these facilitative links, for which the research team was very grateful, this was an independent evaluation, with the York researchers being responsible for all aspects of the design and conduct of all of the WPs, and analysis and interpretation of all of the results.
Analysis of the administrative data set (work package 1)
We experienced four main problems associated with using administrative data for research purposes:
determining availability, receiving the data, merging multiple data sets and understanding what the data really meant. Although these were largely overcome because we could work in partnership with Dementia UK, several challenges remained. These included only partial coverage of all AN services in the central database,
the time-consuming nature of fully understanding and transforming those data into a format suitable for research purposes, and the lack of clarity about what triggered the reassessment of carers’ needs.
We worked to address these challenges, but some remained unresolvable, either completely or in part. So, for example, we could not detect any obvious differences in the types of services that did or did not use the central database, but without data from these services we could not formally test this. Although we worked hard with Dementia UK to ensure that the data we received were as usable as possible, resolving issues around date fields and differing data formats took a long time and reduced the amount of analysis we could then carry out. Similarly, although we also worked hard to understand AN practice in relation to needs assessment, there were no data that explained why needs were reassessed (or not). Finally, the needs assessments did not use standardised tools, so they relied on the nurses’ (inevitably subjective) appraisal of the position of both the carer and the person with dementia.
By their nature, these types of administrative data reflect how a service works and so can limit the analysis that can be undertaken in a research project. Despite this, we were able to provide a summary of the type of clients that the AN service supported, an overview of the interventions that the nurses offered and an estimate of the changes in dyads’ needs over time. Furthermore, we now have a prepared data set that offers opportunities for future multivariate analysis that we had hoped to carry out as part of the project, but which was constrained by the amount of time it took just to get the data into a usable form.
Qualitative work (work packages 2 and 4)
We experienced challenges in recruiting the carers of people with dementia for WP 2. Initially, we had intended to recruit 30 carers through TiDE and to conduct focus groups in York, but we did not recruit the number of carers we needed via this route. In response, we engaged with local community organisations and we were able to identify several peer support groups for the carers of people with dementia, from which we recruited participants. We had originally intended to conduct the bulk of our data collection for this WP through focus groups but, in response to carers’ preferences, we adopted a more flexible approach to data collection depending on individual preferences. Although this responsive and flexible approach meant that recruiting carers for WP 2 took more time than had been anticipated, we recruited more carers than originally planned (n = 35).
A strength of this study was that the survey design was informed by the priorities of carers and their views about which outcomes were likely to be influenced by the services they received. The analysis of the interviews and focus group data from carers fed directly into our choice of outcome measures. Moreover, we were able to test the full questionnaire with a subgroup of these carers, ensuring that the final design was acceptable to them and that carers and researchers had a shared understanding of the meanings of all questions.
We were unable to triangulate findings from the survey and the qualitative interviews with professional stakeholders in WP 4 because there were insufficient survey responses from the case study areas. The number of stakeholders who agreed to participate was also smaller than we had hoped. The pressures of reorganisation, responsibilities for areas other than dementia care and time constraints all seemed to contribute to reluctance among some stakeholders to share their views with us. Despite this, however, we did feel that our material reached saturation in most areas.
The survey and health economics analysis (work package 3)
The main challenge of this part of our work was identifying carers from non-AN areas. We had originally hoped to recruit people from our matched LA areas, using the third-sector organisation TiDE. However, TiDE was, at that point, a relatively new organisation and had not yet been able to rebuild the cohort of carers that its predecessor organisation had access to. We then tried to find carers in our chosen areas via JDR; although this gave us a potentially larger number of carers, using the JDR system was time-consuming. It also identified a fair proportion of carers who were no longer caring at home, because the person they
cared for had entered long-term care or had died; in most cases, this was not evident until carers returned the questionnaire.
We therefore spent much time contacting local organisations in our selected non-AN areas to identify carers’ groups and similar. We also put a link to the electronic questionnaire on some national organisations’ websites, including the Alzheimer’s Society’s Talking Point, and recruited some carers via statutory organisations that had approached us directly. Because of the complexity of the recruitment strategy and our lack of control over how many carers actually received the paper questionnaire or the link to the electronic questionnaire, we cannot calculate an overall response rate. A ‘guesstimate’ of between 25% and 45% is all that we can hazard. The team did finally manage to identify enough carers to have similar numbers of AN and non-AN carers, but the latter were different from the former in a number of important respects, which are discussed in Chapter 5. The variety of sources through which we recruited also made administration of the survey more complicated than we had originally planned.
The survey itself worked well; most carers who responded answered most questions, demonstrating that our chosen outcome measures were, indeed, feasible to use in future evaluation and that carers were willing and able to provide useful information about services that they and the person with dementia used and how much they paid for them.
We think the survey may be the largest, independent, detailed, national survey of carers of people with dementia yet carried out in England. In purely descriptive terms, then, it has value that goes beyond its specific role in this project. The carers we surveyed were, on average, older and more heavily involved in care than those identified in the most recent national survey of carers,6thus giving a unique insight into
a potentially vulnerable group.
In terms of the health economics analysis, and as far as we are aware, this is also the first time that costs and outcomes have been compared between users of specialist nursing services for carers and those receiving usual care.
Typically, within a standard economic evaluation framework, it is useful to collect data over multiple points in time. Although a cross-sectional study may provide a useful insight, it increases the uncertainty about the results because of a higher risk of selection bias (e.g. due to unobserved needs). Even if the IV approach that was used helps to address the selection bias, its estimate of the effect of AN refers to a subgroup of AN carers (i.e. those carers who are estimated to use AN because of the variable contained within the instrument). Other limitations are strictly related to the nature of AN. For instance, the effect on carers who received AN support in the past may be difficult to disentangle from other support services that may also have been used. Diversity in the referral process (in some cases, carers are referred to AN after a triage assessment; in other cases, they can self-refer) across AN providers may generate high heterogeneity within the group of AN carers, which may hamper us in identifying an effect. Finally, we are unable to estimate a summary indicator, such as an ICER, to inform decision-making. Our CQoL measure, although generic to carers, is not generic to all members of the community. In addition, at the time of reporting, no preference weights for the ASCOT-Carer were available. We were unable to calculate a measure akin to a QALY, given that currently we do not have information on time in state or a CQoL preference weight. In addition, based on this, there is no decision rule available to interpret an ICER and no empirical estimate of the cost-effectiveness threshold for decision-making in social care to assist decision-makers.
Informing future practice and evaluation (work package 5)
The AN service we trained to use the chosen carer measures had not had time to implement these into their routine data collection systems before the project came to a close. However, Dementia UK as a whole is keen to integrate these measures into the data collection systems used by all services and we will continue to work with the charity to support this.
Integrating the views of stakeholders to strengthen our findings (work package 6) Work package 6 involved a workshop attended by 36 stakeholders, including carers, practitioners and commissioners. Participants discussed the research findings and fed back key messages and implications. These included overall messages from the research, messages relating to AN in particular and messages relating to the future evaluation of services for carers of people with dementia. We also discussed the ways in which data are collected and used locally to inform and improve services. A full account of the messages from this workshop is given in Appendix 1.
The workshop gave us a valuable opportunity to discuss our initial interpretations of the study findings with professionals and carers with direct experience of the issues under consideration. An example would be the issue of balancing the intensive, specialist nature of AN support (which not everyone requires at all times) with the desire for continuity and full coverage for all carers and people with dementia. There were stakeholders at the workshop who represented organisations that could provide greater coverage than most AN services, but not the intensive specialist support that AN offers, and there was a consensus in the room that these services should work together via a tiered model to ensure continuity and access to specialist support for all. This discussion added weight to our conclusion that, if embedded well into dementia services across a locality, AN could enable the system as a whole to offer appropriate ‘end-to-end’ care and support for all carers and people with dementia.
However, although the feedback from this workshop was useful, we did not feel that it provided a secure base for the production of best-evidence guidelines. Instead, we have produced a short summary of findings, which is now being widely disseminated and is available to download from the project webpage (www.york.ac.uk/spru/projects/admiral-nursing/).