Comparability of data requires that indicators are comparable across organisations or over time based on similarities in functions, processes, methods and outcomes (Nolte 2010). Data comparability is reported as a major challenge when the aim is to compare organisations, especially when comparing internationally (Nolte 2010; Kossarova et al. 2015). Comparison should be made on a like-for-like basis but collecting such data is challenging as various types of care settings provide care differently, have different ways of collecting data and collect different types of data that might lead to similar outcomes (Nolte 2010). Therefore, as Nolte (2010) asserts, many confounding factors can influence comparisons of what may seem to be
51
similar settings, services or organisations. Where there is a lack of comparative data, the
52
process of case-mix adjustment3 can be applied to make the data comparable across organisations (Kritsotakis et al. 2008).
In the context of DCM, one of the important questions is whether DCM does or can provide comparable data that can be used for benchmarking. There is ample evidence that indicates users of DCM believe it is able to provide comparative data. For example, DCM data has been used for internal benchmarking or comparison purposes at individual and group levels across various types of care settings to assess changes in care over time. Brooker et al. (1998) collected DCM data from various types of UK-based care settings, within one NHS Trust, over a three-year period. Comparisons were made to examine changes in care outcomes across nine units (two day hospitals, two respite-care units, four continuing-care units and one assessment ward) over three cycles of mapping. While this study demonstrated a good example of comparing and assessing care changes over time, it has been criticised for not recruiting similar mapping participants in all DCM cycles (Cooke and Chaudhury 2012). However, arguably, while only 25% of study participants remained part of all three cycles, Brooker et al. (1998) study showed that many other important factors were taken into account for credible comparisons. For example, all recruited units had similar models and philosophies of care. This means that all units had single- bedroom accommodation and mixed-sex participants with separate sleeping, washing and toilet facilities and all staff were known by their first names. Further, the staffing ratio, the length of maps and patients’ profiles were similar
3
A case-mix adjustment is a statistical process of adjusting for the differences between organisations and patient characteristics, thus allowing a fairer comparison.
53
supporting effective comparisons (Brooker et al. 1998). This demonstrates the inherent difficulties of comparing service settings over time due to the ever-changing patient/resident population within hospitals and care homes. Therefore, a consideration for benchmarking DCM is the sample representativeness, but also the additional data collected alongside DCM, which permits assessment of comparability or issues which may impact on this.
DCM data has also been used for comparison purposes while assessing changes in care across a number of settings. An American study, assessing quality-of-care in assisted-living (AL) facilities by Kuhn et al. (2002), provides an example of this. Using DCM data, Kuhn et al. compared the quality-of- care provided in both dementia-specific (n=7) and non-dementia (n=3) AL facilities. They used group WIB scores to compare care quality within both types of care settings. While considering specific features of each type of care setting, Kuhn and colleagues also indicate the feasibility of using DCM data to compare AL facilities with nursing homes or day-care centres to assess variations in care. However, this requires additional data (e.g. care settings’ characteristics) collected alongside DCM data to compare similar settings.
The above studies indicate that DCM data can be used for comparison purposes. While all the studies used DCM to make comparisons either across time or between care settings, their underlying purpose was learning and care improvement rather than ranking units against each other, for example by suggesting some were providing good care and others poorer
54
levels of care. This is called comparative benchmarking within healthcare, as will be discussed in detail later.
55
Data comparability is the key to the benchmarking process and one of the major criteria of comparability is to see if data is consistent (Nolte 2010). This means that the collected data should show similar results when collected at different points in time, or within different organisations. Brooker (2005) found in her literature review that various studies had shown a similarity in some of the DCM data collected from organisations providing similar types of care (day-care centres or long-term care centres). Based on this finding, she proposed using DCM indicators (e.g. group WIB score and BCC profiles) for benchmarking. However, she further suggested that confounding factors, such as those related to the participants (i.e. dependency levels) and care- setting characteristics (i.e. type and size), needed to be taken into consideration.
Studies also suggest that residents’ dependency levels may have an impact on the wellbeing of people with dementia. A study conducted by Edelman and colleagues (2004) with participants from special care facilities, assisted living facilities and adult day care centres, found that low mean individual WIB scores are associated with both high levels of cognitive impairment and increased activities of daily living (ADL) dependency. Similarly, another study (Thornton et al. 2004) reported significant correlations between WIB scores and individuals’ total dependency levels and cognitive and behavioural functions. They found that wellbeing levels of those individuals living in continuing care settings and day hospitals are significantly higher, as they had lower dependency levels and fewer cognitive and behavioural issues. Chenoweth and Jeon (2007) also reported an association between lower WIB score and reduced physical function. The DCM Manual also highlights
56
that several aspects could influence the comparability of DCM data when used to
57
assess the change over time (Bradford Dementia Group 2005). For example, participants’ dependency levels and their change over time and data being collected by different mappers are two major aspects, which need careful consideration.
In summary, there is evidence demonstrating the use of DCM data for comparison purposes across time and across organisations. Further, there is some evidence that DCM can produce consistent data across settings offering similar types of care, thus suggesting the potential of using DCM data for comparability.
While data suitability and comparability are important requirements, data needs to be available for benchmarking and to be of a certain level of quality.