Based on the information from the interviews and on face value, it is concluded that most NPT projects are well on their way in the performance of planned activities (output), some staying considerable behind time schedule, others clearly performing above expectations. Physical outputs (like facilities, computers) are of course most easy to assess. However, many projects (also) have less tangible output. Below we present some examples of outputs in NPT that have been generated so far.
Support for institutional capacity development in public and private procurement management (September 2004 – September 2008)
Realised output in following output areas:
• Staff development and training: 3 MBA-ers graduated, 3 will follow; 7PhD-ers on schedule; 30 participants passed NEVI A, NEVI B has started;
• Research and documentation: 6 research proposals on procurement; 2 already approved for funding (after extensive review); research workshops on
design/methodology and data analysis; expert input on research infrastructure and policies; sharing results and data through presentations;
• Course development and curriculum design: review report on existing procurement programmes (at KYU and MUBS); realignment/redesign of
procurement courses has started; curriculum design at KYU for new programmes; development of short courses on procurement, curriculum development academic programmes; review by international procurement professors and experts, partly from abroad; joint development in Uganda by Ugandan universities and same procurement experts;
• Institutional strengthening and cooperation: resource center at Kyambogo Campus will be completed in November 2006; procurement process of 152 computers,
including servers, printers and furniture; procurement of 500 textbooks on procurement & logistics and research completed.
Capacity building of the national council for Higher Education
Realised output in following output areas:
• Research capacity development: (a) trace study (among graduates from higher education) has been completed and is now in the process of being published; (b) government has given some additional funding to enlarge the study (still for additional funding is being looked);
• Information Collection, Selection and Dissemination: (a) resource centre has been established; (b) communication policy has been developed (done in Cape Town, South Africa, during study visit); (c) website has been set up and used for
Evaluation NPT/NFP - The case of Uganda – Field visit report 36
dissemination of information; (d)with funds for equipment, a small but useful printing machine was purchased and is now being used for the production of some
publications in-house; this has improved the quality of publications; (e) stocking of the resources centre is continuing, ICT equipment was purchased and has improved the work of the Council;
• Management Capacity Development: (a) staff were trained in Cape Town (South Africa) last year on communication strategies and Quality Assurance; (b) 6 members of staff underwent 10 days training in management of higher education in
Netherlands in September 2006. Two more training sessions are envisaged in Mozambique before the end of the year and another early next year in Uganda. This is a joint training programme with colleagues from Mozambique (where another Nuffic project is running);
• Quality Assurance: (a)Quality Assurance Framework for the country was discussed at national workshop in April 2005 and June 2006 and finally approved on 28th June 2006; (b) document for institutional self evaluation and audit has been developed; (c) training of 6 universities in self evaluation will take place in third and last weeks of October 2006; (d) training of external academic auditors will take place early next year; (e) by the end of the project, there will be no time to train a second group of
universities in self evaluation. Also it will not be possible to evaluate the impact of this component of the project. An extension is necessary to achieve meaningful results;
• Credit Accumulation and Transfer: (a) A CAT framework paper was discussed at national workshops on 14th April 2005 and 28th June 2006. It was adopted in
principle; (b) minimum requirements for courses of study have been done in Economics, Mathematics, Social Sciences, Statistics, Basic Sciences and Human Medicine; (c) two more disciplines will be dealt with next year; (d) without agreeing on minimum requirements for courses of study, the credit system is not possible in Uganda; (e) an extension of three years is thought to be necessary to have a reasonable CAT system in place.
On almost all of the above mentioned output evidence was shown to the evaluator. The project time of 4 years (NPT) is seen as too short to create sustainable impacts during the course of the project. Extension of highly promising projects should be considered, as well as long term partnerships. However, it should be critically assessed which projects can be seen as ‘highly promising’. Also, measuring impacts is extremely difficult, depending on a large variety of factors that are not in control by the training. Impacts of courses and support as regards capacity building are differentiated:
• Curriculum developments have a high potential for creating sustainable impact at institutional level, if the new curriculum is accepted by all relevant stakeholders in the institute;
• Tailor made training has the advantage that groups of associated people are collectively trained, creating collective commitment, enhancing impact; • Academic courses are more likely to impact primarily on individual level, and
secondary on organisational level. However, it is obvious that 2 or 3 MA or PhD courses cannot reform the system.
As regards NFP/TMT so far hardly any information is available on the results of the courses, let alone the impact. It seems that it is not clear whose responsibility it is to monitor the results. Although it is not requested by DCO, Nuffic has started with
collecting basic information on the appreciation and perceived usefulness of short courses and TMT. Also most training institutes have some sort of subjective evaluation by the participants in the courses. This kind of information so far has not systemically been processed and used in the development of new courses. Measuring impact related indicators is furthermore complicated by the fact that the contacts with participants after the course appear volatile.
In the new Nuffic reporting formats, which have to be filled in by the requesting southern partners, relevance and usefulness are specific subjects which has to be reported on, although from a subjective participants perspective. Nuffic also aims to develop more structural contacts with former participants in the courses.
Of the three districts in Uganda with NFP multi-year agreements two have been visited for the evaluation. The one in Soroti is operational for some time and is clearly positively received by the beneficiary organizations, also due to earlier positive experiences as regards academic and short courses in the Netherlands. The one in Arua is still in the starting blocks, waiting on approval of applications. In Soroti three alumni were spoken. In Arua 5 candidates were spoken.
Results and impacts that are being reported by alumni in Soroti are: • the importance of time management;
• commitment to work/ responsibility; • people management and team spirit; • administration procedures;
• expert knowledge (on adjusted curriculum for children with special need).
A concrete impact that has emerged is that one of the trainees (3 short courses at MDF) is now coordinating the activities of various aid organizations which are working in the region of Soroti. From a perspective of local government this could lead to a more effective communication with these organizations, eventually leading to a better use of the budgets that are available for Soroti.
For another trainee the impact was much less visible. The 3 week course was perceived as too short, and also not everything was applicable in daily work, although this could change in the near future,.
Dissemination is promoted by inviting the trainees to share their knowledge and
experiences with the staff, their colleagues and the wider community in which they work. The alumni’s in Soroti also have a role in informing new NFP candidates.
The impact on individual level is assessed critically by the supervisor developing some sort of performance management. Eventually, this can be a very effective way of assessing impacts on an individual level.
Evaluation NPT/NFP - The case of Uganda – Field visit report 38
In the district of Lira the MYA so far has not been effectively implemented. The various interviewees at the side of RNE and Nuffic could not give any concrete information on the reasons for this. The evaluator sees a clear lack of ownership on the side of the programme management (Nuffic) as well as apparently on the side of the requesting organization in Lira. Also the position of RNE in monitoring the MYA and possibly playing a role in problem solving is not clear.