5.2.1 Types of output measure
The evidence reviewed and responses to our consultation suggested that it is possible to group the types of output measures used in six broad categories.
Vacancy registration
Vacancy registration may have declined in importance in the portfolio of PES roles as they first have taken up a greater focus on activation and second, since the advent of the internet and introduction of private free-to-use vacancy registration services. Nevertheless, most PES continue to register and advertise vacancies as part of a job broking and matching service. The review also suggests that a wide number of PES continue to see this as an important part of their role that needs to be subject to performance measurement. Vacancy registration in some form is therefore a prominent output measure (Cyprus: p84; Estonia: p92; France: p97; Lithuania: p112; Portugal: p120). Austria (see p79), Belgium (p82) and Hungary (p104) go further than simply measuring the number of vacancies and use indicators of the relative quality of vacancies, with these structured according to skills level required in the case of Austria and indicators of permanency in the latter two.
Activation interviews completed
Several PES appear to measure outputs in relation to the number of ‘interviews’ completed with beneficiaries overall and particular groups of beneficiaries (France: p97; Ireland: p108; Norway: p132). While the nature of the interviews completed is not entirely clear, the prominence of such interventions in the performance regime suggests that these can be regarded as ‘activation’ interviews similar to the Work Focused Interview (WFI) interview in the UK where the emphasis is on job search coaching, confidence building and the promotion of active job search, for instance through the provision of Better off in Work Calculations. In Norway a target is established for this indicator and in France the measure is an ongoing one of whether these are undertaken on a monthly basis.
The measurement of such interviews helps to establish a number of reference points for management interpretation of performance. To the extent that such interviews are regarded as important in ensuring and promoting active job search and transitions into employment and they are the subject of substantial organisational investment then their measurement is a central indicator of whether
26
the organisation is undertaking the work activity required of it. To the extent that this data can be linked to sub-national off-flow or other outcome data, then it may be reasoned that causal assumptions of what types and volumes of activity help to generate desirable outcomes. Measurement of such interventions can also serve as an indicator of workload pressure on the organisation and therefore may provide contextual information which might help to accurately interpret outcome trends. In Great Britain, the Interventions Delivery Target (IDT) is intended to work in conjunction with Job Outcome Target (JOT), with the former measuring the timeliness of work-focused interventions and the latter providing a form of quality check of the effectiveness of those interventions. However, with other exogenous factors influencing the level of job outcomes that flow from interventions, there is a need to support this model with other ways of measuring quality and effectiveness.
Referrals to training/activation provision or other support
Referrals to training, external provision or other support to overcome barriers to work is a central aspect of active labour market programmes (ALMPs). It is, therefore, unsurprising that many PES measure this activity as an output indicator. These include Belgium (LeForem) (p82); Czech-Republic (p87); Denmark (90) Estonia (p92); Germany (p100); Hungary (p104); Ireland (p108); Lithuania (p112); Norway (p132); Portugal (p120); Slovenia (p126); Sweden (p130). In some cases these include a rapidity indicator (e.g. Sweden) and in other cases they are specified to particular benefit groups (as in the case of Norway and several indicators and targets related to interventions for lone parents). In several of these cases there is also a differentiation between different types of ALMP referral.
Referral to programmes is a useful indicator for many of the same reasons as interview completion – it helps to understand the volume and nature of activity and support being provided to job seekers. It also helps to contextualise off-flows and specifically off-flows after particular types of interventions and, where it is possible to combine this information with off-flow indicators and to disaggregate this information between different types of programme then this can help to understand the different success rates of different types of intervention. Again though, volume measures don’t provide a complete picture and the quality of referrals plays an important part in the success of the process and the different programmes and interventions.
Penetration measures
In some cases referrals to provision or other types of intervention were measured as a proportion of some other population or over a particular time period to constitute a penetration indicator.
Individual plan completion
Several PES use output performance measures related to the completion or processing of individual statements/plans: Though the precise nature of these were not clear they appeared to take two forms. The first was a portfolio of information Commentary on an international review of performance management systems
27
which appeared to relate to either benefit claim processing or referral to private sector provision (e.g. in the Netherlands). In other places this appeared to be similar to Jobseekers Agreements (JSAg) which set out work-related aspirations and planned activity and support that job seekers will take/require in order to fulfil these (as in Portugal: p120; Slovenia: p126; Spain: p129; Sweden: p130). Again, like referrals and interviews, where individual action plans are part of the package of ALMP interventions assumed to make a difference there is some rationale for monitoring data on their timely completion.
Placement into work trials or subsidised employment
In three countries placements into work trials or subsidised employment appear to be monitored as performance indicators in a similar way to referrals to training and external provision (Estonia: p92; Slovenia: p126; US: p140). What is important to note in both of these cases is that these indicators are both used as part of a package of other indicators.
Sanctions
Despite many of the PES in our review apparently applying more rigorous expectations of benefit recipients and a more activation-oriented approach with sanctions and conditionality, only Slovenia appeared to be tracking sanctions as a performance indicator.
Process quality measures
Several countries match output measures with more qualitative measures of process quality. These are presented here under output measures as they are closely related to outputs. For instance in some PES (e.g. Netherlands: p116) process quality measures are directly related to output measures such as the comprehensive completion/quality of individual dossier/plans or the timeliness of the delivery of key interventions as discussed above. A further category of process quality measures relates to ‘customer feedback’, whether this be from individual job seekers/benefit recipients or from employers, usually gathered in survey form (Netherlands: p116; Sweden: p130; US: p140), though in at least one case this is matched with a system of quality standards (Norway: p132).