III. MATERIAL Y MÉTODOS
3.8. Procedimiento de lavado y sembrado 2) Paquete de
module
As well as monitoring the day-to-day activities of your advocacy strategy, you also need to keep monitoring your progress towards achieving your objectives.
The purpose of evaluation is to assess how effective your advocacy has been in bringing about the changes intended, as well as any unintended changes, and what can be learnt from the process to make your future advocacy more effective.
There are many ways to evaluate your impact.The best method to use will depend on what you need to know, who needs to know it, how much rigour is required, and the resources available.You may need to carry out an independent external evaluation, or it could be an internal process.The important thing to keep in mind is that evaluation is about learning – what you learn should change what you do. Here are some of the more common methods:
Quantitative methods, including baseline and follow-up surveys
You can measure change by doing a quantitative survey to compare the situation before and after you began work. For example, you might compare information on attendance rates at school by children from a minority ethnic group before the date your advocacy work started and afterwards.You can also collect survey information on the levels of knowledge of people being educated and stakeholders’ attitudes. But be aware that surveys can take a lot of time, skills and money, and if they are not carried out well their validity is limited.
Qualitative methods, including key informants and focus group discussions
You can also use a qualitative approach such as the judgements of a small number of people who are in a good position to assess your impact. Such ‘key informants’ (possibly including your targets or influentials) can give subjective judgements about how effective you have been. For example, journalists may be able to give you feedback on how the media is interpreting the importance of your message; bureaucrats may give you insight as to whether politicians’ opinions are changing. Focus groups enable you to gather several views at once, and to stimulate a discussion about the outcomes of your advocacy. Matters of judgement are as important as objective assessments, and an evaluation should usually include both.
Involve the audience and include childrenin monitoring and evaluating your work in order
to share learning. Involving allies and key informants during the evaluation itself is one of the best ways of promoting this sharing process. Some of your audience may also be better placed to make neutral judgements about the effectiveness of your work.
Share your results.Even if your audience is not directly involved in your evaluation process,
you need to think about how best to share the lessons you have learned. For example, you could publish a case study, or make a presentation at an appropriate community meeting.
Timing of the evaluation.You should time your evaluation at a point that will enable you to
feed the learning back into your planning and help motivate you to keep going, for example, after one year of a three-year advocacy strategy.The evaluation process can also be part of your advocacy strategy, as it can help strengthen relationships with allies, targets and influentials, and give them an opportunity to feed their views back into the process.
9.2 Evaluating the outcomes of advocacy
9.2
156Participant’s manual
module
Outputs # of meetings # of hearings # of attendees at a hearing # of people who sign up for the list
# of workshops # of participants
The following framework sets out indicators of progress you might see on your way to achieving the more substantial change objectives, and the impact of your advocacy.
A very simple model:
module
9
Start point Increased dialogue (early outcomes) Changed expressed opinions (intermediate outcomes) Changed policy (advocacy objective) Change in peoples’ lives (impact)
Example
Set out your activities in the first column, and your change objectives in the final column. Then consider all the ways in which you expect to see progress on the way.
Measuring change
You can assess change using quantitative and/or qualitative measures.The best method to use will depend on the resources available. See Save the Children UK’s Advocacy
Toolkitfor examples of the kinds of indicators that may be relevant for monitoring
ç
ç
ç
ç
Advocacy activities Meetings with policy-makers and officials on target issue Testimony at a hearing Set up a constituency email list Advocacy training workshop Indicators of short-term and intermediate outcomes• Change in % of influentials who believe your message about target issue (survey) • # of co-sponsors on a bill Organisation’s statistics used in a formal meeting summary
Increased # of people taking action to send letters to elected officials
Change in individual members’ skills, capacity, knowledge, and effectiveness
Indicators of long-term objectives and impact
• Changed policy or legislation • Policy/legislation
implemented
• Increased governmental funding designated for topic area
• (and in the very long term) positive change in people’s lives as a result of the policy/legislation change (impact)
Evaluating your advocacy work and any resulting impact on policy change is complex. There are a number of issues to bear in mind as you design your evaluation and interpret the results:
Complexity:The policy arena is complex and ever changing. There are many steps
along the way, which also need to be measured and celebrated, such as constituency building, research, public awareness, meetings and building relationships with officials, etc. Success is hard to achieve and issues around the needs and rights of children often face serious cultural restraints.That is why incremental changes should be valued – they represent building blocks that help an issue move to its next stage.
Role of external forces:Players and dynamics outside of your control often greatly
influence policy and advocacy work. So it is possible to do everything in your control “right”, and still not achieve your final goal because of external factors (e.g. a change in party legislature).
Time frame:Policy change goals are often long-term.Thus, although a policy might take
many years to pass and be implemented, there will be many steps along the way to influence and shape the policy environment.These are important in terms of your overall advocacy strategy and should be measured as benchmarks along the way.
Shifting strategies and milestones:Advocates must continually review goals and
strategies to fit the changing environment.
Attribution:Many different actors are involved in advocacy and it can be difficult to
attribute policy change to the work of any one player. Attributing credit for bringing about change can also be political, especially when working with government partners or other civil society organisations. The best policy is to share credit.You can demonstrate to your constituencies the specific actions and influences Save the Children contributed to bringing about change, but sharing credit with others is an investment in being able to work together on future challenges. In public – a speech, press conference, radio interview, or public meeting – sharing credit is the best and safest policy.
Real versus apparent change:Advocacy may bring about apparent change without
making a difference to children. For example, a change in policy may have no further impact if it is not implemented or enforced. Your evaluation should look at what has really changed as a result of advocacy. What has improved for children, what stories illustrate this? What more has to be done? If your advocacy strategy has achieved no change then you need to learn important lessons for future advocacy work. Creating real change for children is a long distance effort that requires stamina and persistence.
Direct and indirect outcomes.While policy change is often the main objective of an
advocacy strategy, evaluation should look at other outcomes, for example, increasing the advocacy capacity of civil society groups, or increasing children’s participation in decision- making.Your evaluation should also ask some key questions to improve the effectiveness of any future activities. For example, how has the advocacy affected your relationship with major targets? How has it affected the strengths and weaknesses of specific coalitions? How has it affected your reputation with different audiences? How has it affected broader perceptions of the issue?