The choice of a mixed-methods design incorporating a survey and case study has enabled the team to investigate contemporaneously and in depth, the processes for selecting and applying QIs in community
nursing quality assessment.95Despite the initial problems of obtaining data from the new CCGs, our survey
in relation to variation in organisations providing community nursing and the number and range of local QIs in use for community nursing in 2014/15. Inferences made about suspected workload involved in respect of selecting and monitoring quality measures based on the survey findings were able to be explored in the case study. The use of five health settings has facilitated comparison of findings across different types of community services provider organisations. Our within-case analysis suggested very little variation in contextual characteristics impacting on how community nursing service quality is measured within the different organisations. This is possibly due to the strong policy-drivers from NHSE for delivery of care closer to home creating some turbulence by pushing forward organisational change and increasing integration in all the sites, following on from the implementation of CCGs in 2013. Although the generalisability of findings from case studies could be argued to be limited, this study has incorporated triangulation into its data collection methods by conducting front-line and meeting observations, which can both inform and validate our interviews conducted with participants in the field. A dissemination phase enabled us to test and validate our findings with stakeholders from around the country, thus reinforcing confidence in our findings.
We purposively selected case site organisations from scrutiny of the findings of the survey, and
consequently there may have been some differences between those sites that agreed and those that did not, which might have influenced the case study findings. Two sites we approached initially seemed to agree to participate, but in one case took so long to provide requisite information for the NHS permissions process it had to be abandoned. In another potential site that included a private provider, despite a great deal of enthusiasm from its nursing lead, we were unable to recruit as, at the final stage, a high-level decision meant that the site suddenly withdrew. It is possible that market sensitivity and concerns about confidentiality were behind the decision to withdraw. This experience highlights a number of issues that need to be considered with the wider involvement of private providers in this area of provision. First, claims of the need for confidentiality of commercially sensitive information may make it increasingly difficult to scrutinise the activities of companies in order to understand lessons from which other organisations can learn. Second, an unwillingness to take part in research (which is essential for NHS improvement) will be problematic unless a requirement to support NHS-funded research is made explicit. NHS organisations are themselves subject to a clear expectation that they will support research as a core activity.
It is recognised that people who volunteered to be interviewed or shadowed in the case sites may have been different from those that did not allow access. However, our study design incorporated a dissemination element that enabled our findings to be shared with a range of stakeholders across the country to test out the extent to which our analysis and interpretations resonated with staff and patients in different locations around England. The research team considered the effort and resources expended for this valuable exercise justified, in that these events validated our findings and produced valuable insights into how they could be translated into actionable good practice guidance, all of which significantly enhanced the strength of our study and its outputs. The use of all the above methods of triangulation and validation considerably strengthens the trustworthiness of the findings.
New CCGs were not quite embedded in 2014 when the national survey was undertaken, and this was the reason for a pragmatic decision to use FOI processes as a data collection method. The use of FOI ensured that the organisation found the correct source of information and advice, as telephone operators
rarely knew to whom they should address our enquiry, a problem that has been reported elsewhere.95
Furthermore, utilising the knowledge of our commissioner co-applicant, and in order to increase our response rate, rather than ask people to complete a long questionnaire, we asked them to send us their documentation relating to quality of community nursing in 2014/15. Requesting documentation designed for work purposes resulted in receipt of a variety of documents at different stages of completeness. The most consistently complete data reported the CQUIN scheme and, as our design incorporated an ensuing case study, affording the chance for in-depth analysis of other quality measures, we focused on CQUINs for the survey. This possibly narrowed our focus too closely on this aspect of service quality measurement; however, negotiating and monitoring the scheme proved to be a key area of work for commissioners and enabled insight into the competing quality priorities between commissioners and service providers.
The focus on CQUINs also highlighted the difference in emphasis given to front-line concerns and the national agenda. As the scheme was incentivised it also offered the opportunity to add to the meagre literature on the unintended consequences of incentivised care in relation to community nursing. We were able to scrutinise a range of supporting documentation on other measures from commissioners and providers, including front-line staff, although this was comparatively limited owing to variation and incompleteness in the documentation obtained. Commercial sensitivities following the introduction of competitive commissioning may also have influenced what documents were made available to the team. Owing to the prevailing wider circumstances relating to staff vacancies and a high degree of change impacting on community nursing services, we were sensitive to ensuring that front-line staff shadowing was timed for the convenience of organisations, missing the winter pressures and only commencing when the research team had built a relationship of trust with nursing team leaders. Nevertheless, one of the sites, the social enterprise, did refuse to allow shadowing of staff without giving any explanation, although front-line staff were able to participate in a focus group. As the findings of that focus group were in accord with focus groups in different sites, it is arguable whether or not there would have been different issues emerging in relation to front-line observations but, again, commercial sensitivities may have been behind the decision to deny access. In the other four sites, community nurse team leaders allocated staff for shadowing and in several cases this was a more senior member of the team. Although this might have influenced the findings of observations to an extent, it did enable a more in-depth discussion of care context and quality than might otherwise have been the case.
Lack of parity in documentation received meant that it was difficult to identify any meaningful differences in our cross-case analysis of this aspect of the data collection. The main difference identified was that one case site (a combined acute and community trust) had chosen not to participate in the 2015/16 CQUIN scheme. It was unclear whether this was due to a pragmatic decision to minimise financial risk or because it did not prioritise community services. Additional documentation, such as retrospective quality accounts, was complete and accessible online. In theory these reports enable patients and carers and other stakeholders to access information about service quality. In reality, although quality accounts do provide evidence of provider services’ achievements against quality targets, accompanied by comments from the local Healthwatch organisation, the reports have the appearance of a public-focused marketing brochure (their structure and appearance differed
across the case sites, again making comparison unfeasible). Furthermore, the‘marketing‘ of quality accounts
highlights the problem of linking QIs to financial awards and penalties, as it is bound to lead to organisations seeking to market themselves and present the most positive face possible, rather than openly highlight and learn from problems in order to improve them.