5. MARCO METODOLÓGICO
5.4 CARACTERIZACIÓN DE LA POBLACIÓN
5.4.3 Etnografía virtual
As discussed inChapter 1, identifying opinions and commentaries for a realist synthesis is the first stage in identifying theories for which evidence is later sought. The purpose of this search is to map the range and diversity of different programme theories underlying PROMs feedback, rather than identify and include every single paper discussing the ideas and assumptions underlying PROMs feedback. We conducted one search for programme theories for PROMs feedback at both the aggregate and individual level. Opinion pieces and commentaries on PROMs feedback were identified in database searches, JG’s personal library (89 known relevant studies) and citation tracking activities including forwards and backwards citation searching.
In April–May 2014, we searched the following databases:
l Cochrane Database of Systematic Reviews (via Wiley Online Library), issue 5 of 12, May 2014
l Cochrane Methodology Register (via Wiley Online Library), issue 3 of 4, July 2012
l Database of Abstracts of Reviews of Effects (via Wiley Online Library), issue 4 of 4, October 2014
l EMBASE Classic+EMBASE (via Ovid), 1947–30 April 2014
l Health Management Information Consortium (via Ovid), 1983–present
l (Ovid) MEDLINE®, 1946–week 3 April 2014
l (Ovid) MEDLINE®In-Process & Other Non-Indexed Citations, 1966–29 April 2014
l NHS Economic Evaluation Database (via Wiley Online Library), issue 4 of 4, October 2014.
Two search strategies were run on the Ovid databases: one aimed at identifying review papers and one aimed at identifying commentaries and opinion pieces. The Cochrane Library databases were searched with one strategy to identify reviews only, as they were unlikely to contain opinion pieces. The searches were developed iteratively; initial searches developed by the information specialist (JW) were discussed with JG and SD, who provided feedback on whether or not useful papers were being captured. JW then revised the search strategy.
All search strategies included search concepts for PROMs and the‘Outcomes of Feedback’. Subject headings and free-text words were identified for use in the search concepts by JW and project team members. Further terms were identified and tested from the personal library (known relevant) papers.57
Care was taken to avoid retrieving papers that simply reported PROMs outcomes, and to identify those with discussion of the feedback of PROMs.
An‘opinion pieces’search strategy from a previous realist synthesis,89conducted by the same authors,
was tested against the known relevant papers and used (with a minor adaption in MEDLINE to include the search term‘comment.cm’). An example of the PROMs feedback‘opinion pieces’search is presented in Appendix 1. The search strategies for review papers used the Clinical Queries–Reviews specificity
maximising filter in Ovid databases plus a series of specific free-text searches to identify reviews (seeAppendix 1).
The database searches identified 1011 references, which reduced to 837 when duplicates were removed. These records were stored in an EndNote (Thomson Reuters, CA, USA) library alongside the 89 personal library references to create set of 748 references.
JG and SD screened the titles and abstracts of the 748 references to identify potentially relevant papers according to the following criteria.
Inclusion criteria
l The paper describes how aggregate PROMs feedback is intended to work.
l The paper provides a theoretical framework that describes how aggregate PROMs feedback is intended to work.
l The paper provides a critique of the ideas underlying how aggregate PROMs feedback is intended to work.
l The paper reviews ideas about how aggregate PROMs feedback is intended to work.
l The paper provides stakeholder accounts or opinions of how aggregate PROMs feedback does/does not work.
l The paper outlines, discusses or reviews potential unintended consequences of aggregate PROMs feedback.
Exclusion criteria
l The paper reports findings in which a PROM is used as a research tool [e.g. an evaluation of an intervention, a study exploring the health-related quality of life (HRQoL) of specific populations].
l The paper is focused on evaluating the psychometric properties of a PROM.
l The paper reviews the psychometric properties of a PROM or a collection of PROMs.
l The paper provides advice or recommendations about which PROM to use in a research context. An initial screen identified 94 potentially useful papers; the titles and abstracts of these papers were then rereviewed by JG and categorised according to the different theories they articulated. All of these papers contributed to the process of mapping the programme theories underlying PROMs feedback. Following this process, 46 were selected for inclusion, as they provided the clearest examples of the ideas underpinning the feedback and public reporting of PROMs and performance data. These papers represented the same ideas and assumptions contained within the full set of 94 papers and, in essence, were a purposive sample of these papers. The full texts of these papers were then read, together with an additional 30 papers from JG’s existing library, which included key policy documents and grey literature. Notes were taken about the key ideas and assumptions regarding how the feedback and public reporting of PROMs data was intended to work. Of the 46 papers identified from the literature searches, 15 were purposively selected as‘best exemplars’of the ideas reflected in the papers as a whole. These were cited in the final draft of the paper cataloguing and summarising the programme theories underlying the feedback and public reporting of aggregated PROMs data, reported inChapter 3. However, again, they represented similar ideas and assumption as the 94 papers initially selected for inclusion. Forwards and backwards citation tracking of key articles, and additional, iterative searches in Google Scholar (Google, Inc., Mountain View, CA, USA) as key subtheories emerged, identified a further 30 papers that were cited in the final draft, reported inChapter 3. In all, 75 papers contributed to the development of programme theories. Regular discussion of these ideas among the project group, circulation and feedback of draft working papers outlining the theories ensured that the full range of different theories were represented.Figure 1summarises the flow of studies from identification through to inclusion in the final document.
Alongside these searches, we also held informal meetings with a number of stakeholders. These included the insight account manager for PROMs and a senior analyst from NHS England, an information analyst for the national PROMs programme from the Health and Social Care Information Centre (HSCIC), a GP commissioner from a Clinical Commissioning Group and a NHS trust lead for patient experience. In these meetings, we explored stakeholders’views on how the national PROMs programme and the national surveys of patient-reported experience measures (PREMs) were intended to improve patient care. These interviews were neither tape recorded nor formally analysed. Rather, they were used to clarify our ideas about how these programmes were intended to work, and to support and expand the programme theories we identified from the literature.