TIPO DE ESTUDIO

The terms of reference are an important part for the research methodology in the second segment of this thesis. The second segment, which has an explanatory approach, focuses on asking 'why' and 'how' questions through semi-structured interviews with stakeholders and experts associated with the policymaking and agenda setting process of the Netherlands government its contribution to the United Nations Mission in South Sudan. This evaluative approach is based on the terms of reference framework from the evaluation and guideline set for evaluations of the Policy and Operations Evaluation Department of the Dutch Ministry of Foreign Affairs (2009). Kingdon (2011) already noted the importance of feedback for the executive branch of a government to signal certain problems in policy areas. Kingdon also noted that feedback can also serve as a process to evaluate and oversee the implementation of a certain policy to formulate possible adjustments. The terms of reference toolset offers a framework for this thesis to ask these 'why' and 'how' questions to gain more insight into several aspects of the policymaking process of the Netherlands government. This paragraph will first consider the terms of reference, before discussing the inclusion of critiques on evaluation exercises by Peter Swanborn (1999), who was written extensively on this subject.

The evaluation and guideline set for evaluations

The Dutch Ministry of Foreign Affairs includes an independent evaluation body which is the Policy and Operations Evaluation Department. The objective of this department is to increase insight into aspects of the implementation and its effects of Dutch foreign policy (Ministerie van Buitenlandse Zaken, 2011). Although most of the evaluations are carried out by this department, other evaluations can be carried out by individual policy departments or embassies abroad.

The operational definition of the concept of evaluation which is maintained by the Policy and Operations Evaluation Department (2011) is "an evaluation is a systematic and objective scrutiny and assessment of a policy, policy instrument or activity. An evaluation should identify the results achieved, thus giving an indication of effectiveness and efficiency, as well as the factors that led to these results". This is also in line with the definition used by the Organization for Economic Co-operation and Development's Development Assistance Committee (1998), which highlights the importance of the systemic and objective nature of evaluation exercises to determine the efficiency and effectiveness of a certain policy. This definition highlights the importance of evaluations to gain insight into certain policies, instruments or activities to enhance the decision making with regard to certain policies. Evaluations also enable all ministers a degree of accountability to the House of

Representatives for its policies as well as the allocation of resources. The importance of budgetary requirements and value acceptance are important factors that determine the future feasibility of a certain policy (Kingdon, 2011). The evaluations also serve as a vehicle for targeted feedback to improve and adjust policies in the future to strengthen its effectiveness and overall implementation (Ministerie van Buitenlandse Zaken, 2009).

The next paragraph will discuss the terms of reference, which will be used as the main methodological fundament for the evaluative approach in the second segment of this thesis. The subsequent paragraph, which contains important remarks with regard to evaluation exercises from Peter Swanborn, is used to further complement the terms of reference in terms in terms of academical applicability.

Terms of reference

The terms of reference are the steering mechanism behind the evaluative framework of the Policy and Operations Evaluation Department and establish the conditions under which an evaluation is to be conducted. According to the evaluation and guideline set for evaluations, the terms of reference should contain important elements such as the reason and justification for the evaluation, its aim, core questions, the scope, the approach and choice of methodology, a level of representativeness, a time path, budget and the expected products or reports that will be published (Ministerie van Buitenlandse Zaken, 2009:30-31). The terms of reference also notes a number of prescribed questions that must be taken into account for a policy review, and these questions are of interest for this thesis. These questions (Ministerie van Buitenlandse Zaken, 2009:31) are as following:

• What problem gave rise to the policy? Does the problem still exist? What is the cause of the problem?

• Why does the government consider tackling the problem as its responsibility? Why is the responsibility at central government level (and not at local or EU level)? What form does that responsibility take and why?

• What objective has the government formulated to solve the problem?

• What instruments are being deployed? How coherent are they? Is there overlap? What is known about the implementation of the policy and the efficiency?

• What is the effect of the instruments on the formulated objectives (solution to the problem)? • How are the budgets determined? On what basis?

These questions have been redrafted to be used for the semi-structured interviews, and linked to Kingdon’s theory of multiple streams. There are a number of important elements in the questions

above that can immediately be linked to the streams, especially the question with regard to problem recognition and the formulation of objectives and policies. The aspect of the effect of the instruments on the formulated objectives is important to the fourth subquestion, to what extent the Netherlands government its intentions with regard to the contribution to the United Nations Mission in South Sudan is connected to its capabilities. This question takes an evaluative angle which is the main goal of the second segment of the thesis. The next paragraph will discuss important critiques and remarks from Peter Swanborn on conducting evaluation exercises which will be included to complement the terms of reference as well to enhance the set approach for the interviewing process.

Swanborn’s remarks on evaluation research

Peter Swanborn has written extensively on research methodology and empirical evaluation research. In Swanborn his book Evalueren (1999), the methodological basis and systematics behind evaluative exercises is provided, with emphasis on the evaluation of government interventions from the general principles and guidelines for scientific research. This paragraph will discuss some important remarks on conducting evaluative research that is applicable to the second segment of this thesis, which takes on an evaluative approach to assess the Netherlands its contribution to the United Nations Mission in South Sudan. These remarks are important because the terms of reference are not developed by an independent academic body, but by a governmental institution. The incorporation and adaptation of some of these remarks are important to increase the applicability of the terms of reference in an academic setting and as a research methodology for the second segment of this thesis.

One of the first remarks of Swanborn is that in general, evaluation exercises take insufficiently place. There are numerous examples of government interventions, with little to no comprehensive insight into its intended effects. In addition, due to the continuous development of government policy and institutional governance, the demand for autonomous evaluation is vital to understand more about its effectivity and efficiency (Swanborn, 1999:28).

The second remark by Swanborn is about transparency. Full disclosure of all policy strategies is often in nonconformity with policy objectives, especially in certain policy areas such as defence and foreign affairs. It’s therefore important to involve stakeholders that are involved in the policy process to increase the relevance and quality of the research itself (Swanborn, 1999:31).

The third important remark is one of the necessary conditions in Swanborn’s framework for evaluative exercises. It relates to already existing policy interventions. One of the most important

elements of an evaluative framework is the question, if, and why, existing policy interventions are inadequate in tackling the originally specified policy problem (Swanborn, 1999:46).

Another important aspect of the framework is the question in what way the policy has to be adjusted to achieve the formulated policy objectives. This is an important aspect, which cannot be found back in the terms of reference of the evaluation and guideline set for evaluations from the Policy and Operations Evaluation Department of the Ministry of Foreign Affairs (2009). This is an important remark for the second segment of this thesis, which attempts to evaluate through the interviewing process to what extent the implementation of the policy solution is addressing the problem it was designed to address.

The fourth remark concludes the previous remark and is also important for the second segment of this thesis. According to Swanborn (1999:64), for policymakers, concepts such as quality is mostly rhetorical. It serves to ensure the general public that the policymaker is still engaging in his or her actual task, while the actual agenda is something different. It is thus the question to which extent the government as a client of the evaluative exercise is interested in the results, especially if it neglects the policy objectives that are being pursued. Swanborn (1999) notes that it’s important to take into account that if actors have invested heavily into certain a policy, both in financial terms as well as prestige — a negative return of investment will be perceived with reluctancy by the general public. An evaluative exercise should therefore not shy away from inquiring into all aspects of a certain policy, whether they could have a negative outcome or not.

The last remark that has a degree of importance for the second segment of this thesis is an advice from Swanborn who states that every evaluation researcher should know that, while researching government interventions, they are operating in a political context (Swanborn, 1999:331). Academic principles in terms of objectivity and credibility should therefore be respected with the highest regard and steps should be taken to triangulate all sources from both governmental and non-governmental sources in the interviewing process to minimize political influence and the politicization of discussed policy strategies.

This paragraph discussed the terms of reference as well as important remarks from Swanborn on evaluation research. Both elements are important for the research methodology for the second segment of this thesis, which is aimed at assessing to what extent the Netherlands government its intentions with regard to the contribution to the United Nations Mission in South Sudan is connected to its capabilities. The next chapter will discuss the literature review concerning the inception of the Mission, as well as changes in the Netherlands government its contribution.