HISTÓRICO
PRECISIONES TERMINOLÓGICAS
choose appropriate evaluation instruments which can be used to collect evidence both from project workers and from target groups. Experienced project co-ordinators may have favourite and familiar evaluation techniques which have worked in the past. However, every project is different and it is also important for projects to keep up to date with recently developed method- ologies and be imaginative in the choice of evaluation tools. The following table could be used as a means of narrowing down the choice and also showing where a single tool could be used for several purposes:
Update partners on what has happened
■ ■
Focus on specific aspects of the project to be monitored
■ ■
Be based on already organised and interpreted information
■ ■
Be clear about the key documents and sources of the infor-
■ ■
mation
Reflect critical points in the project cycle (milestones)
■ ■
It is important to remember that these should also be occasions when project achievements are celebrated!
‘Critical friend’:
This term is open to interpretation but usually refers to an individual who is asked to provide an informal view of project progress from time to time. ‘Critical friends’ will usually not enter into a formal contract in the same way as an external evaluator.
Cross project evaluation:
This is a practical proposition when two or more projects are engaged on similar themes or working with similar target audiences. The two projects each devote some evaluation time to the other project. As the evaluators are involved in similar projects they understand very well the thematic field and the structural framework of the activities. There may also be considerable savings in the planning stage since both projects can benefit from the same documentation. Cross project evaluation also brings other benefits since the two projects will become linked and a certain amount of cross fertilisation of ideas and techniques is inevitable. Cross-evaluation might be cheaper than contracting an external expert. Within the Lifelong Learning Programme there is also the potential for the collaborative development of both projects as a network.
Agile and Open Space – the possibilities offered by new methodologies:
The Agile system of project management evolved in the world of software development. This system is based on flexibility, on working groups and teams coming from diverse backgrounds and not necessarily using a hierarchical approach to the man- agement of their project. In terms of project evaluation this implies that a range of evaluation tools need to be available to the group to be used appropriately.
Open Space Technology is connected with meetings and events of the sort that are typical of a Multilateral Project.
A comparative analysis of folk tales: a multicultural perspective (Comenius Project)
The partner meetings in this project were basically planned in order to evaluate methodologies and results. During these meetings the partners shared their own experiences along with the dissimilar and diversified practices they experienced. Moreover the partners attended the workshops run by their colleagues whenever possible during these meetings. Since the consortium was composed of academicians/scholars who were from different disciplines – although all are related to education – the applications and practices were enriched by suggestions and re-structured for the best results. As well as being a multicultural study, the project has also been operated on a multidisciplinary basis. In this way both the partners have re-evaluated and enriched their own activities, workshops and applications under peer inspection within an internal observa- tion system and they have also contributed to the improvement of the activities and their applications in a reciprocal way. Project website: www.caft.gazi.edu.tr
Quality
and
Evaluation
Chapter
9
subsequent decision making can be seen to be based on a mix of information.
Qualitative data tends to be descriptive text and includes inter-
views, the minutes of meetings or focus groups, case studies, reflective diaries and observations.
Different kinds of data
Evaluation methodologies will inevitably generate data of one sort or another. Therefore before evaluation starts it is important to predict the nature of that data and how useful it will be in steering the work of the project. It is advisable to generate both qualitative and quantitative data, in this way
Box 10: Evaluation instruments and guidelines developed specifically for European projects Box 9: Evaluation instruments to choose from
Project objectives Indicators of change Evaluation tools
1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6
1 Questionnaire 2 Learning Log 3 Web Tool
4 User Group 5 Stats Analysis 6 Interviews
Self-Evaluation in Adult Life Long Learning (SEALLL):
www.sealll.eu/
This project was funded under the Grundtvig strand of the Socrates programme and ended formally in 2007 although material is still being added to the website. In terms of con- tent it firmly relates to the Lifelong Learning Programme and especially to Adult Learning but it is important to note that it is aimed at organisations rather than multi-national projects. As the project title states the focus is on self evaluation, therefore this site does not aim to provide a ready made set of evaluation instruments which can easily be adapted to generate quantita- tive material. The project team envisaged self evaluation as a developmental tool which can be applied by learners, teachers, developers and project managers to aspects of an institution or organisation (learning, teaching, programme, organisation and management and external relations).
A key part of the site is the ‘Tools Grid’ which enables access to a whole series of evaluation instruments. It is very much in
the spirit of self evaluation that a project actor can use this grid to select an appropriate instrument dependant on the target group. Much of the material has been translated into several languages so that target groups can self evaluate in their mother tongue, this is an important factor in qualitative work where a lack of facility with a foreign language can easily take away the meaning of a useful comment.
INTERtool: www.intertool.eu
The INTERTool project aims at providing European project managers and teams in adult education with the basic specific intercultural competencies necessary for successful transna- tional co-operation in the framework of European projects, with a strong focus on the virtual dimension. To this end, virtual and face-to-face learning and training tools are being developed, produced and tested. INTERtool includes an Evaluation Tool alongside its other aids to project management.
(cf. also Chapter 7: Intercultural Elements in European Project Management)
Quality
and
Evaluation
Chapter
9