Using the judgement analysis approach, a professional’s judgement reliability can also be examined. By including repeated scenarios, judgements of the repeated instances can be compared to those given when the scenarios were seen initially. For example, Jacklin, Sevdalis, Harries, Darzi and Vincent (2008) gave 30 trainee surgeons a series of case scenarios of patients who required a cholecystectomy (operation to remove the gallbladder), and asked them judge the likelihood that they would choose to change from conducting keyhole to open surgery based on information about the patient’s condition. The participants were shown 64 case scenarios, of which 20 were repeated. The correlation between judgements of risk of the repeated cases versus when they were first judged ranged from 0.47 – 0.98, with a mean of 0.77. This was seen as an indicator of the degree of variation between participants in their judgement consistency. This type of analysis is useful because it provides an indication of the expertise of different professionals.
In the context of the financial abuse case scenarios, if participants adopted a consistent approach to decision making they would be expected to made similar judgements when faced with a repeated scenario, including their certainty of abuse, likelihood of action and selecting what actions they would take. Consistency of actions selected was measured using the following formula, developed by grant holder KG:
Action consistency = Total no. of matched actions*2 Overall sum of actions
‘Total no. of matched actions’ was a measure of how many of the same actions participants chose in response to a scenario at both original and repeat presentation. For instance, for the social care and health professionals this was from a maximum
of 90 (15 repeated scenarios * 6 actions). The ‘Overall sum of actions’ was the number of actions each participant selected across the original and repeat scenario sets, from a possible total of 180 for the social care and health professionals (30 scenarios including 15 original, 15 repeat * 6 actions). Researcher MD generated action consistency scores for each participant in SPSS. Higher consistency scores (from a possible range, 0 - 1) indicated participants who tended to select the same actions in response to a repeated scenario.
An additional measure that can be used in judgement analysis research to assess the expertise of professionals is the Cochran-Weiss-Shanteau (CWS) index (Weiss & Shanteau, 2003). The CWS index was developed to measure expertise in situations where the accuracy of judgements could not easily be determined. Expertise is assessed based on the judgements made, rather than any external criteria (Weiss & Shanteau, 2003). This could be applied in relation to financial elder abuse decision making, given that there is no clear criteria for assessing if professionals’ have correctly identified suspected abuse. The CWS index represents a ratio of discrimination between judgements of different situations, versus consistency when faced with identical situations. Higher CWS scores indicate good discrimination and low inconsistency, therefore suggesting a high level of expertise (Weiss & Shanteau, 2003). CWS is measured as follows:
CWS = Discrimination Inconsistency
The CWS index was used by Rassafiani et al. (2009) to compare the expertise of occupational therapists when determining the management of children with cerebral palsy who were experiencing upper limb hypertonicity (extreme tension). Occupational therapists (n=18) viewed 110 case scenarios, which included 20 repeated scenarios. They had to select one of seven options for the management of each case, which were presented in increasing level of invasiveness. CWS scores were calculated for each participant to determine how consistent they were in their judgement of the repeated cases, as well as their ability to discriminate between cases. Cluster analysis of the CWS scores identified two groups of participants, one labelled ‘high performers’ who showed better discrimination and consistency than the ‘low performers’ group. Rassafiani et al. (2009) reported that there was no significant difference between the two groups in terms of their years of experience, and used this to evidence the need to focus on applying measures in addition to years in job role to determine expertise.
The CWS Index was thought to show a good application to judgements in relation to financial elder abuse, to provide an additional means of distinguishing between professionals performance. This would complement analysis of judgement consistency, and cluster analysis with consideration of the demographic characteristics of different cluster groups. An illustration of the steps taken to generate discrimination, inconsistency and CWS scores in Phase II can be found in Appendix 6.8. The scores were calculated following the explanation of the measures provided in CWS: A User’s Guide (k-state.edu/psych/cws), as well as the explanation in Friel, Thomas, Shanteau and Raacke (2002).
6.3 Summary
• The methods employed in Phase II were described in this chapter. In Phase II of the research, a quantitative approach was chosen to investigate the relationship between the cues of financial elder abuse, and how professionals made judgements of certainty of abuse and the likelihood that they would take action.
• A factorial survey approach was selected applying a fractional factorial design to present professionals with financial elder abuse case scenarios containing different combinations of the cues of financial elder abuse.
• The participant sample was made up of social care professionals (n=70), health professionals (n=82) and banking professionals (n=70).
• Social care and health professionals completed a separate Phase II online task to the banking professionals.
• Multiple regression analysis with dummy variables was conducted to explore the influence of the financial elder abuse cues on professionals’ judgements of certainty of abuse and likelihood of action. To supplement regression analysis, t-tests were conducted to compare the impact of each cue category on judgements, and incremental f-tests were calculated in order to assess the overall impact of each cue.
• Professionals’ action choices were examined to consider the ‘strongest’ action selected as well as the most common action choice. The relationship between certainty of abuse, likelihood of action and action choices was also considered.
• In order to assess the impact of demographic characteristics on decision making, a cluster analysis based on raw judgement scores was conducted to compare the characteristics of different cluster groups.
• Judgement consistency was measured by considering the correlations between judgements of both certainty of abuse and likelihood of action in response to repeated scenarios. Consistency of action choices in response to repeated scenarios was also examined.
• In order to assess professionals’ level of expertise, CWS Index scores were calculated to measure how well professionals could discriminate between different situations as well as their ability to make consistent judgements.
Chapter 7
Phase II results: Modelling financial
elder abuse cue usage by social care and health
professionals
This chapter presents the findings from Phase II of research with the social care and health professionals. In Phase II, participants were shown financial abuse case scenarios, with the aim being to determine how the cues of financial abuse incorporated in the case scenarios influenced their judgements. The results show how professionals used the financial abuse cues to reach judgements about the cases. The judgement policies of individual professionals are explored, as well as distinctions between social care and health professionals and consideration of cue usage at an overall level. Findings regarding the impact of the decision maker’s characteristics on judgements are explored, as well as the degree of judgement consistency shown. In the discussion section, the results are evaluated in the context of previous literature considering implications for the detection of financial elder abuse by social care and health professionals.
7.1 Introduction
In Phase II of research participants were presented with case scenarios about suspected financial elder abuse and were asked to make judgements about each case, as if they had encountered it in their working life. The aims of this phase of research were to determine how professionals used the financial abuse cue information when making judgements, and to explore whether demographic characteristics had any influence on judgements made. Evidence from analysis of professionals’ judgements could then be used to identify those who showed a consistent approach to decision making. The Phase II research questions were as follows:
1. Which case features (cues) explain the greatest variance in decision making by professionals?
2. Do participant demographic characteristics show any relationship with decision making?
3. Which participants are the most consistent decision makers?
7.2 Results
Descriptive findings from analysis of the social care and health professionals’ judgements of financial abuse case scenarios is presented prior addressing the Phase II research questions.