Trajectòries de rendibilitat de l'exemple

Main aim: did intervention activity address issues identified from the ‘screening’ phase?

The key aim of the planning phase was to ensure interventions were tailored to the needs identified in the baseline assessment. These three initiatives were clearly linked to the survey findings, and aimed to address some of the main issues but, crucially, not all.

177

Demands/workload-related issues were acknowledged by PublicOrg in the summary they shared with staff, but no activities addressing it were put in place during the research period. This is also discussed further in the chapter conclusion.

‘Implementation’ phase

Main aim: maximise the ‘reach’ and ‘quality’ of interventions

Implementation ‘reach’ (i.e. employees’ exposure to interventions) and quality was assessed by the questions in the T2 survey that asked whether employees had experienced each intervention and, if so, how they rated it (positive, negative, or neither). The effects of exposure to and perceptions of interventions is the subject of more detailed analysis in the following ‘effect evaluation’ chapter. But on the basis that wider intervention ‘reach’ (i.e. employees’ exposure) was indicative of more successfully disseminated organisational interventions and that ratings were an indicator of better implementation, these can give a general gauge of how effectively interventions were implemented.

Based on the T2 survey (n = 1,008), 76% of respondents reported having experienced ChangeComms, while 75% indicated they had experienced the new PDF approach to supervision and appraisal. Although this clearly indicates that not all employees had been exposed to these interventions, it had reached a large proportion of the workforce. The mental health awareness sessions had been attended by 9% of the sample, although these were more not aimed at the whole workforce.

In terms of how these interventions were perceived by employees, 41% of attendees rated the mental health awareness sessions as positive, with 8% negatively. The T2 survey asked attendees for their comments on the mental health awareness sessions and generally these were also positive and indicated it was good to raise awareness. For example;

178

- “It reinforced that the steps I have been taking to support staff is appropriate” (participant T2:181)

- “It has helped and given me greater awareness around mental health issues which I can now go onto share” (participant T2:346)

However, in determining why these sessions may not have ‘reached’ more managers, a small number of respondents felt they did not need such training. Similarly, some others who did attend felt the material was too basic and did not meet their needs, although PublicOrg’s aim for these sessions was to be an entry-level introduction for managers who lacked that knowledge. Tellingly, finding the time to attend was also problematic for some, as was awareness of the sessions. For example: -

- “I have not been made aware of these briefings and would not have had time within my diary to attend even if I did know” (participant T2:309)

Although employees’ exposure to PDF and ChangeComms was much wider than the mental health awareness sessions, as they were intended to be, the ratings were more mixed. ChangeComms was rated positively by 28% of respondents, while 18% rated it negatively; PDF was rated positively by 22% and negatively by 17%. The majority, therefore, perceived PDF and ChangeComms as ‘neither positive nor negative’.

The follow-up survey included a further open-text question asking for employees to comment on the baseline findings and/or the action taken by PublicOrg since the previous survey: ‘Do you have any comments about the results of the last survey, or the actions taken by PublicOrg since the last survey?’. There were just 60 comments provided in response to this question (i.e. 6% of T2 sample provided a comment; summary of main themes shown in table 18).

179

Table 18: Summary of employee comments about survey findings & subsequent actions

Category

Main theme of comments

‘Do you have any comments about the results of the last survey, or the actions taken by PublicOrg since the last

survey?’

Scepticism/disappointment about actions

(20 comments)

• Actions resulting from the previous survey were not directed at the main causes of stress-related problems

• Actions haven’t addressed the main issues/points from the survey

• Nothing done to improve the stress/pressure staff are under

• Did not know what had been done since the first survey

• Nothing would come from the survey/results and that it was a token effort/tick-box exercise

Promotion/awareness of survey & results

(18 comments)

• Survey results were extremely delayed

• Survey results were hard to find/’buried’

• Did not know about survey results

• Would like to have seen full report, rather than summary

Survey issues

(15 comments)

• Survey length (too long)

• Concern over anonymity and why some questions were asked

• Survey is ok, but staff need more of a say

o E.g. focus group, working group, staff drop-in sessions

• Survey is positive

Accuracy of survey findings

(12 comments)

• Results as expected

• Shows widespread issues – staff overworked

• Does not show the extent of staff unhappiness

Other

(8 comments)

• Several comments answered other questions. E.g. most stressful issues, or suggestions.

This represents a small proportion of the sample, but comments tended to focus on the survey findings themselves. However, where comments did reference the actions taken by PublicOrg, it was generally negative or reflected a lack of knowledge of actions.

- “Staff are under a great deal of stress and pressure. Nothing has been done to improve this” (participant T2:452)

180

- “Don’t feel anything has been done to address the issues it [the survey] raised” (participant T2:542)

- “The issue is a scepticism around action taken as a result of the feedback given, and a general lack of communication /engagement / leadership at the top of the organisation” (participant T2:433)

Furthermore, there were no qualitative comments specifically referencing either PDF or ChangeComms in response to this question. This was perhaps surprising given there were both positive and negative ratings given by respondents to the closed questions about them. This is explored further in the discussion, as there were both methodological and process-related lessons stemming from this. However, PDF and ChangeComms were mentioned by comments in response to other questions.

- “I did enjoy the mental health awareness and budget meetings” (participant T2:224)

- “The new appraisals scheme with its apparent focus on progression may be a positive change, it feels too early to say though” (participant T2:986)

However, few of these comments were positive and they tended to relate to how as much as what was implemented. For example, in relation to PDF: -

- “New appraisals - no explanation offered just introduced“ (participant T2:180) - “The change to the PDF was muddled and unclear about when it should have been

implemented” (participant T2:16)

- “Make sure supervisors are carrying out regular supervisions and appraisals (PDF) as this does not happen on our team’ (participant T2:30)

181

This suggests that at least some employees had not connected the actions to the first survey, and the survey feedback, and that at least some managers were not kept informed about it, and that implementation was variable – as evidenced by ‘only’ 75% of employees reporting they had experience of PDF.