Conclusiones

I. Introduction

In 2015, the JIU embarked on a review of “fraud prevention, detection, and response in the United Nations system”. The main objective of the review was to inspect the fraud control management programmes and processes of United Nations system organizations and the implementation of anti-fraud policies and procedures in allowing effective prevention, detection and reporting of fraud. One of the data collection instruments used was that of a web-based anonymous confidential survey sent to all staff of the 28 JIU

participating organizations, as well as a second confidential survey sent to the senior-most managers200of

the JIU participating organizations.

The surveys allowed triangulating data and information gathered through other data collection methods, such as relevant literature review and analysis, formal questionnaires, extensive person to person interviews with management and staff at various organizations, reviews of findings, conclusion and recommendations, including status of follow-up, by internal and external oversight bodies (internal audit, external audit, investigation, inspection, evaluation), audit advisory committees, as well as assessments by other key functions, such as ERM, finance, procurement, etc.

To assure an unbiased elicitation process with high quality data for a reliable analysis, particular care was given to the methodological approach employed in the design, data collection and analysis of the perception surveys. The methodology included several systematic steps: survey design, testing, necessary sample size computation, implementation and data collection, verification and control, and survey analysis.

II. Survey design

The survey design underwent several revisions to improve the elicitation process, overall survey structure, and data collection. The questions included in the perception survey were carefully prepared and revised by the project team and supported by a data scientist (statistician) external consultant. The team also drew upon fraud-related surveys conducted by some of the participating organizations’ oversight offices and other international entities, as well as selected all staff surveys conducted periodically by some of the participating organizations’ ethics and/or HR offices.

In order to have a reliable response rate and to avoid respondents’ fatigue, questions were limited in number, and the overall survey length was kept as short as possible (tests carried suggest that respondents could complete the survey within 15 minutes). There were a total of 30 questions for the all-staff survey, and 10 for the senior managers’ survey.

The survey question structure was composed of six sections. The first section requested demographic information from survey participants e.g. which institution they work for, whether they work at the headquarters or country offices, contract type, years of service, level, and functional group type. The remainder sections prompted questions aiming to elicit participant perceptions on their institutions’ anti- fraud culture and fraud awareness; anti-fraud policies and programmes; prevalence of fraud, fraud risks, reporting fraud and whistle-blower protection; sanctions and disciplinary actions; and Improvement of anti-fraud policies and programmes. This structure supported analyses of relevant aspects, issues and subjects related to fraud prevention, detection and response.

The response choices to questions included: agree, partially agree, neither agree or disagree, partially disagree, disagree, with an additional option to indicate “I do not know.” Some questions required a ‘yes’ or ‘no’ response. In three cases respondents were asked to provide a ratings on a scale from 1 (“no improvements needed”) to 5 (“major improvements needed”); this allowed to providing ratings on for example fraud risks, and areas for improvement. There was an open question (text box) which allowed respondent to provide clarifications or to bring up any issues as they see fit. Except for the first section of demographics (questions 1-8), respondents were able to skip questions. The questions on demographics allowed to filter responses (e.g. general staff versus professional staff; headquarters versus field staff; different function groups, e.g. ethic, legal, programme, etc.), including combining different filters and criteria for a more granular or detailed survey analysis.

Some questions of the all staff fraud survey were also used for the senior managers’ survey, in view of allowing for some comparison and cross-analysis. Both surveys included an open question (text box) which allowed respondents to provide any clarifications or to bring up any issues as they see fit.

III. Determining the response sample size and the validity of response rates

In order to draw reliable inferences about fraud perception at the different organizations of the United Nations system, appropriate amounts of information were necessary. Determining the sample size involved several qualitative and quantitative considerations. Qualitative factors considered were: the nature of the research, the number of questions, the answer choices per question, sample sizes used in similar studies, incidence rates from similar surveys administered by the JIU, completion rates, and resource constraints.

The sample size was determined as the actual number of persons required from a population of interest to make the survey analysis as representative as possible of the entire population. Important quantitative considerations in the determination of the sample size were: population size, confidence levels, the level of precision, and the degree of variability. The population size is the total number of people we would like to draw inferences from. In this particular survey, the number of employees per UN organization was used as a proxy for population size; the confidence level was a measure of how certain the researcher is that the sample is representative of the population within the range of precision selected the margin of error was the measure of how close the sample’s answers were to the “true answer” from the population. The targeted number of respondents per organization was computed based on the criteria: 95% confidence level and 5% error margin. The sample size computation was carried out for each UN organization using the following formula:

ܵܽ݉ ݌݈݁ܵ݅ݖ݁=

ݖଶ_{∗ ݌(1 − ݌)}

݁ଶ

1 + ൬ݖଶ∗ ݌(1 − ݌)_݁ଶ ൰

Where p is population size, e is the margin of error, and z-score is the statistic relative to the confidence level chosen.

The below attached table presents by organization and for the United Nation system as a whole the confidence levels and margins of error actually obtained by the two surveys.

IV. Verification of the statistical validity:

In order for the JIU to draw reliable conclusions the survey, it needs to be representative of the population of interest, present a high confidence level and a low margin of error. In particular, in line with above sample size considerations, a suggested sample size was computed according the target population of each

individual institution to be surveyed. The computations were made based on two target confidence levels, at 95% and 90% and error margins of 5% and 10%. For the survey distribution and data collection, the JIU teamed up with focal points in each institution to collect as many survey responses as possible. It then was verified whether the JIU reached the per organization desired number of responses (as suggested by the confidence levels). At the same time, organizations survey samples with appropriate confidence intervals and margins of error (i.e. 95 % and +/- 5 %) were identified.

After the closure of the all staff survey, at the United Nations system level a total number of 15,929 staff responded and the confidence level was high at 99.26%, with the margin of error being very low at 0.76%. At the per organization level the highest confidence level obtained was 98.32% with a 1.65% error (WFP), while the lowest confidence level was 83.61% with 16.06% error (UNRWA). Except for the latter organization, all other organizations presented a confidence levels above 90% and a maximum error of 8.9%.Therefore reliable conclusions at both organization and global level are obtained (except for UNRWA).

In the case of the senior managers’ survey, at the United Nations system level, a total number of 164 managers responded and the survey presents a 94% confidence level and a 5.79% margin of error. Only two organizations (UNHCR and UNOPS) reached the targeted minimum 90% confidence level (with ~0% error). Reliable conclusions can be reached from the senior managers’ survey at the global level. For details please refer to below table on confidence levels.

References:

 Yansaneh, I.S. (2003). Construction and use of sample weights. United Nations Statistics Division.

 Levy, Paul S. and Stanley Lemeshow (1999). Sampling of Populations: Methods and Applications. Third edition. John Wiley & Sons, New York.