EFECTO DE LA DISFUNCIÓN DE LA CRM SOBRE LA EXPRESIÓN DE IL-

The literature review has surveyed examples of technical discourse (TIEK) and socio-political perspectives on development impact evaluation, i.e. critical configurations. These are two contrasting domains of knowledge and understanding about contemporary data/knowledge intensive impact evaluations in the development 2.0 landscape. The review of both helps us to sharpen the focus of the study, to move from TIEK prescriptions and critical configurations towards how power is entangled with data/knowledge, within impact evaluation practices. TIEK data/knowledge is predominantly targeted at and circulates around professional and administrative audiences in development bureaucracies, networks and markets (Wallace et al, 2006; Mosse, 2004a; Quarles von Ufford, 1988; Ilcan & Philips, 2008). This flow is reversed for data capture, where data is primarily captured from intervention target communities, clients, households, business owners, and local workers (Hyman & Dearden, 1998). Thus, the evaluation input/output information meta-model separates data suppliers and data consumers, constituting a professionalized data flow (Ilcan & Philips, 2008), raising questions about the value of evaluation broadly (Barr et al, 2016), and its less clear value for beneficiaries (Duncombe, 2009; Wenar, 2006)15_{. Both scientific evaluation discourse focused on}

effectiveness, attribution, and empirical robustness, as well as participatory discourse foregrounding local empowerment narratives are evident in TIEK prescriptions.

The critical configurations illustrate how the TIEK method plus results frame and information centric input/output meta-model have repercussions for small NGOs, evaluation agents and evaluation relations. The configurations involve small NGOs, frontline data/knowledge work, and the evaluation power/data/knowledge silences, transformations and inequalities that emerge and become sustained in practice. From the review, three implications emerge, the first two as foundational concerns for the following chapters, and the third a caveat about the wider the corpus of literature.

The first implication is that the history, diverse demands and evaluation configurations constitute a broader range of issues than those confronted by TIEK. TIEK’s normative prescriptions produce results that are primarily understood as information to be used in rational

15 _{There are calls to make the evaluation processes more open and provide more feedback to beneficiaries (e.g.}

decision-making processes within aid organisations. The breadth and depth of demands and configurations mean that TIEK provides an insufficient response to the problems of power/data/knowledge relations in development impact evaluation. Methods don’t explain broader configurations and do not account for aid chain practices or power. Results don’t acknowledge power or practice in bureaucratic flows, market competition, or how impact representations are shaped outside of professional standards and empirical data. The target community is the focus of the evaluation gaze, and yet the results are not intended for target communities, but for professional communities, aid chain agents, and decision-makers, all of whom are almost invisible in the TIEK audit gaze.

Key issues were raised in the critical literature around how small NGOs deal with critical configurations around impact evaluation work.

Regarding evaluations, how do small NGOs:

• deal with evaluation issues beyond target sites, TIEK prescriptions, or evaluator skills?

• make space for diverse voices beyond expert problem-solution frames?

• account for hidden politics or how methods change over time?

• balance globally sourced methods and local ways of knowing impact? Regarding NGO work itself, how do small NGOs:

• tend to the needs of distant contexts and stakeholders versus local ones?

• navigate communications, branding and marketing needs?

• manage expert technical knowledge?

• balance evaluation office work versus field work? Regarding politics and governing, how do small NGOs:

• weigh compliance and market rationalities against relationship building?

• balance the construction of mobile data/knowledge for professional communities versus local stakeholders?

• navigate technical results and anti-politics pressures alongside political negotiations, with local agents and aid chain agents?

The second implication that emerges from the literature, is the need for a perspective on evaluation that privileges organisational, social and political practices. Such a view of evaluation is a) critical in terms of acknowledging broader socio-economic power dynamics embedded within data/knowledge construction processes, and b) engaged in terms of recognizing and responding to, not just interpreting from a macro, geographic, scholarly or sectoral distance, what evaluation data/knowledge does, at different points along aid chains. Interpreting without engaging risks creating more governing, market or colonizing demands and prescriptions for agents such as small NGOs.

This implication raises a question about three established evaluation perspectives and supports further investigation of a fourth. The first perspective concerns evaluation as technical or scientific. The second addresses evaluation as participatory, but still prescriptive, targeted and often technically configured in practice. A third concerns critical views of evaluation which articulate arguments for evaluation reform. However, there is a case for a fourth perspective which positions evaluations as critique plus engagement, simultaneously, partially and in situ. Mohan (2001) identified the problem that underlies this tension between critical and engaged views. He acknowledged the twin dangers of intervening and of passivity, in the face of:

“… our common subjugation to increasingly global material forces and the possibility of transformative dialogues [which] - make the need for alternative forms of collaboration urgent and pressing (Mohan, 2001: 167).

Following Mohan, and keeping contemporary development 2.0 shifts, opportunities and the configurations discussed above in mind, how can an approach be formed for both critiquing and engaging with impact evaluation power/data/knowledge dynamics? Such an approach requires a deeper discussion of conceptual and methodological frames, the focus of the next two chapters.

However, before embarking on the theory and approach chapters, it is important to recognise a third implication from the literature. This is that there exists an extensive corpus of further literature which deals with the study of and analysis of data, information and knowledge in diverse scientific and socially informed ways. This corpus spans areas such as the philosophy of information and knowledge for example, or technical work on data analytics and data science. The philosophy of information for example draws on older empiricist philosophy and theories of mathematics in order to understand what information is (e.g. Adriaans & van

Benthem 2008a,b; Lenski 2010; Floridi 2002, 2011). Data science and data analytics foreground how data is analysed, interpreted, stored and communicated, drawing on disciplines such as statistics and computer science (e.g. Tukey, 1962; 1977; Cleveland, 2001; Press, 2018) Of closest relevance in this wide body of literature is work that problematises power or inequalities. One example of this is work by Floridi (e.g. 2014; 2015), which analyses how explosive developments in contemporary Information and Communications Technologies (ITCs) have influenced who we are and how we relate to each other, our organisations and our economic sectors, from entertainment, to education, healthcare or even personal conduct. The fact that organisations now face challenges and difficulties “all essentially driven by the recording, transmitting and processing powers of ICTs” (Floridi, 2014: 168) means our production, consumption and mediation of data, information and knowledge, in evaluation or other such information intensive processes, requires serious attention. Floridi sees the circulation of information and knowledge as forms of power, wherein “power is informational, and exercised through the elaboration and dissemination of norms” (2014: 176). This contrast with power as a more physical, authoritative force.

Such “norms” amplified through the circulation of information and knowledge are a key aspect of the critical configurations discussed earlier in the literature. These shape the legitimacy of processes such as evaluation, its professionalised methods, acceptable results, relationships, and so forth. In evaluation, methods and results become desired by particular professionals, but inequalities or damage accrued due to the adoption of new data, knowledge or information norms, can remain unclear, even silenced. Floridi advocates not normative ethics per se as a way of dealing with these problems, but the design of information processes and systems which embed the need for ethical choices and reflection. He calls this an “infrastructure” for ethics, “infraethics”, or pro-ethical design, contrasting it with ethics by design, and gives an example about organ donations (2015: 190).

“For example, strategies based on ethics by design may let you opt out of the default preference according to which, by obtaining a driving licence, you are also willing to be an organ donor. Strategies based on pro-ethical design may not allow you to obtain a driving licence unless you have indicated whether you wish to be an organ donor: the unbiased choice is still all yours.” (Floridi, 2015: 190),

Activities such as obtaining a driving license, or donating one’s own organs are well understood in terms of what decisions need to be made, when, and where. However, it is less clear how development impact evaluation systems and processes can be mapped and made more infraethics compliant – it may involve many choices at many sites of evaluation design, data collection, analysis and reporting for example. Where should ethical choices be built in and how should they be enforced in development evaluation? A more detailed map of the micro- practices of evaluation and the power relations involved, who makes what decisions when, who collects, extracts, uses or benefits from evaluation information for example, is required first, before an infrastructure for ethics can be designed.

The next chapter focuses on how data, information and knowledge are related with power dynamics in impact evaluation, in order to understand the socio-political aspects of evaluation in practice. It does not cover the breadth of the extensive corpus in information science, information studies, the philosophy of information or the technical prescriptions of data science and data analytics for example. However it does offer an alternative to contemporary models of data, information and knowledge which have, as will be argued, first of all become implicit and widespread in development evaluation practice, and secondly have become agential in evaluation power/data/knowledge relations, having marginalised the critical importance of power and politics in evaluation data, information, and knowledge construction practices.