1 Determinants of the Legitimacy of Performance Assessment among Public
Officials in Colombian Subnational Governments Alejandra Rodas Gaiter
Performance assessments are one kind of management tools that seek to
induce desirable behaviors among public employees. They also help guarantee
individual and organizational expectation alignment, create knowledge about
development, and allow agency control. However, in countries still in process of
professionalizing their civil services, the performance measurement appears more
as a formality than a strategic process that brings valuable information to
management. This characteristic reduces the legitimacy of the performance
assessment among public employees and supervisors, leading to a vicious circle in
which performance is not assessed properly nor generates positive incentives
towards greater performance. Our research question explores what are the
determinants of the perceptions of legitimacy of performance assessments among
Colombian public officials. To answer this question, we use a qualitative approach
to analyze some semi-structural interviews. In addition, this study uses data from
one large survey in Colombia. We find that the role that managers play and the
alignment have a positive relationship with the legitimacy of the process, whereas
INTRODUCTION
Performance assessments can be a potent tool to induce desirable behaviors
among employees (Ayers, 2013; Kim, 2010). According to the literature it has,
among others, three main purposes: -knowledge development and agency control
(Daley, 2005), -goal alignment, and -behavioral change (Kim, 2010). As an
organizational tool, the assessment informs about staff performance and enables
the evaluator to offer proper feedback (Kellough, 2012; Kim, 2011). It brings
valuable information about the employees and the way they are doing their
activities. Thus, the assessment of the employees does not only provide
information about performance in itself, but becomes an incentive for more (or less)
productive behaviors.
Furthermore, performance assessment can contribute to goal alignment
between the individual and the organization. This alignment occurs when the
performance assessments ensure accountability and results-oriented outcomes
because they link the employee’s activities with the organizational goals (Ayers,
2013). Goal alignment is vital to increase organizational performance, as Andrews,
Boyne, Meier, O´Toole and Walker (2012) demonstrate in their study. It enhances
performance assessments’ value and empowers its role in organizational and
individual development.
Moreover, the performance assessment can induce behavioral change in
employees. The very measurement of performance may help to modify employee's
behavior in a performance management context (i.e. Hawthorne effect) (Kim,
2010). Furthermore, when employees perceive that organizational rules are
3
increase. This change in behavior can be translated into better individual
performance, which in turn enables the achievement of organizational objectives.
Therefore, anyone in charge of human capital management policy in the public
sector should consider this and other unintended effects from the evaluation of
performance.
Performance assessment systems have evolved to be better understood as a
way to improve organizational performance in the public sector. Employees react
and develop personal interpretations of the organizational rules and environment,
and eventually, these issues affect their behavior and performance in the
organization (Wright, Moynihan & Pandey, 2012). Consequently, performance
assessments can be considered a way to socialize employees about the rules and
incentives of the organizational system. Additionally, performance assessments
are also useful for the organization to obtain feedback about the employees’
commitment and goal alignment with the performance administration system.
Nonetheless, there is a gap between the expected results of the assessments
and the actual results the organizations achieve. According to the extant literature,
this gap is due to: -structural problems in performance management systems and
instruments of measurement (Kellough, 2012; Randell, 1994), -the role that play
the stakeholders who participate in the evaluation process (Andrews et al., 2012;
2012; Kim 2011) and -the cultural and communication difficulties inherent to the
process (Ayers, 2013; Kellough, 2012). These aspects have also been identified by
prior scholarship as crucial factors in explaining why performance assessments
The factors that explain failure have been poorly studied in most countries
still in process of professionalizing their civil services. For instance, in Colombia a
law was enacted in 2004 to develop a comprehensive system of performance
measurement in the public sector and still, ten years later, there is a large
implementation and legitimacy gap regarding performance assessments in this
country. The situation seen in Colombia is not uncommon to most developing
countries on similar stages of civil service development (Iacovello & Strazza,
2014:36; Longo & Echebarría, 2014:64). In these places, the performance
measurement appears more as a formality than a strategic process aiming to bring
valuable information to the organizations management. Moreover, the evaluation
system receives little recognition from key actors involved in the process as a tool
for organizational and individual improvement (Strazza, 2014:29).
Given the situation previously described, this article attempts to explore the
issue of legitimacy of performance assessments among Colombian public officials.
Our research question is: What are the determinants of the perceptions of
legitimacy of performance assessments among Colombian public officials1? We
consider that answering such question will provide valuable information regarding
the actual factors that hinder the process of evaluation inside public organizations
and reduce its degree of adherence by the subjects of evaluation. It can also
inform about the implementation of the assessment system itself and about the
1 We focus on public officials at the subnational levels of government. Since 1991 Colombia is, according with
the Constitution, a decentralized unified State with autonomous territorial entities. This was a main point of change, where the new system aimed to vest the subnational governments with financial, administrative and political decentralization. The current arrangement brings to subnational governments (Alcaldías y Gobernaciones, in Spanish) the implementation responsibility of the key public policies such of health and education.
5
effectiveness of socialization practices that the agencies use in the global
management process to make this an incentive for better performance.
This research endeavor has two main goals. First, by exploring the legitimacy
of the performance system among public officials, we aim to observe how
employee perceptions affect the actual effectiveness of this management tool.
Second, we aim to provide a better understanding as to how a performance
system, and its related imagery, does operate as a competitive framework that can
elicit higher (or lower) performance behaviors among employees.
To answer our research question we develop both a quantitative and a
qualitative approach. Our unit of analysis is the individual, namely public officials at
the subnational level. For the quantitative part we conduct an ordinal probit model,
since our dependent variable (legitimacy perception) is categorical with more than
two options. The data comes from the Survey of Institutional Environment and
Development, which covers more than 19,000 public officials and is conducted
every year by the Colombian Statistical Department (NSD). We also use the Index
of Municipal Performance developed by the National Planning Department (DNP),
for information regarding the administrative capacity of the subnational
governments. In the qualitative analysis, we focus our attention in the human
resources managers’ perception about some key aspects of the relationship of
interest and also in the quantitative results.
This work offers three main contributions. First, it provides an empirical study
of the performance assessment in the context of a country still striving with the
professionalization of its civil service. This can bring important elements for the
system in a less studied context. Second, according to the latter, the results can
enlighten policy improvement and highlight the aspects where the assessments are
weak at the subnational level of government. Third, the findings can help to better
understand, through empirical evidence, the factors that have hindered the
advancement of a proper implementation system in non-Western settings.
The structure of this article is as follows: in the first section we review the
relevant literature regarding performance assessments and employees'
perceptions of their legitimacy. We explain how any evaluation system should be
perceived as legitimate to be effective, and how structural and cultural
determinants affect this perception. In this sense, performance management
strategy, measurement instruments, goal alignment, manager role, and the cultural
context explain the differences between the expected and given performance
assessment results. Then, in the second section, we provide some contextual
information about the performance assessment and their legitimacy in public
organizations in Colombia. The third section shows the hypothesis, methodology,
the data and our statistical approach. We move forward to show and discuss the
principal results in the fourth section. This work finishes with some concluding
remarks and some policy recommendations.
I. LITERATURE REVIEW AND THEORETICAL FRAMEWORK
There is a growing recognition in recent empirical scholarship about the
value of the performance assessments as a tool for improving performance (Ayers,
7
Murphy & Cleveland, 1995; Nigro & Kellough, 2014; Prowse & Prowse, 2009).
Performance assessments have an important role in performance management
systems because of their ability to induce desirable behaviors among employees
(Ayers, 2013). Similarly, they are also instrumental in the design and structure of
Human Capital Management (HCM) policies and other related issues such as
employee wellbeing and career development (Daley, 2005; Kellough, 2002;
Murphy & Cleveland, 1995). Those characteristics, besides helping the execution
of the HCM strategy, bring legitimacy to the Human Resource decisions and help
improve the organizational performance.
However, for an assessment system to work properly and contribute to the
global organizational strategy, evaluations are to be perceived as legitimate by
employees. As the scholarship has shown, when an employee perceives
performance-oriented rules in his organization as fair and trustful, his commitment
and motivation with the organization mission will increase (Kim, 2010). Therefore,
there is a positive relationship between the organizational performance and public
officers’ views regarding the performance systems.
Having said that, there can be a disconnection between the expected results of
the performance assessments and the actual results of the organization´s
performance (Delic & Cebic, 2011; Agbola, 2011; Nigro & Kellough, 2014; Prowse
& Prowse, 2009). This disconnection could abate the legitimacy of the
assessments among key stakeholders. In fact, it can eventually reduce the
effectiveness of the organizational performance strategy and put in risk the whole
implementation of the assessment process. Hence, it is assumed that any
actually effective. Thus, according to the little literature regarding individual
performance assessment in the public sector, it makes sense to explore how the
legitimacy of these processes among public servants in a country like Colombia,
affect the entire management strategy of public organizations.
It is important to define the legitimacy concept that we use in this study. As
Suchman (1995) explained, legitimacy is an anchor point of the normative and
cognitive forces that constrain, construct and empower organizations. It is a notion
with a multidimensional character that must be interpreted differently in diverse
contexts (Cruz-Suárez et al., 2014; Suchman, 1995). According with Bouckaert
model (1993) and Streib and Poister (1999) interpretation, an effective
performance measurement system must be legitimate, especially to the lower level
employees. This characteristic strengthens the measurement system but also
provides the necessary resources to continue and grow.
Scholarship in this topic has delineated different definitions of legitimacy (Box,
2002; Cruz-Suárez et al., 2014). In this case we are going to conceive legitimacy
as the “generalized perception or assumption that the actions of an entity are
desirable, proper, or appropriate within some socially constructed system of norms,
values, beliefs and definitions” (Suchman, 1995). This definition incorporates the
strategic vision of the subject from the management perspective, but does not
neglect its own cultural influence of the institutional view. It also includes the
Fambry and Harper (2005) definition of the legitimacy as the process of stabilizing
the organizations and activities that are often the foundations of the bureaucratic
9
and processes (micro) rather than the general analysis, concentrated on
organizations and governments as a whole (macro).
In conclusion, legitimacy affects how people act and understand the
organizational dynamics. It is a social construct that starts from the congruence
between behavior and the set of shared beliefs by a social group (Box, 2002;
Suchman, 1995). Therefore, the legitimacy as a perception is possessed
objectively but is created subjectively (Suchman, 1995). This concept and its
interpretation are important to organizations in order to make sense and construct
value but are also essential to winning acceptance in the strategic audiences.
Previous scholarship has attempted to identify the operational elements
affecting the views of employees on performance management systems. Some
identified issues of the performance measurement process can be grouped into
three main types: a) structural problems in performance management systems and
their instruments of measurement (Delic & Cebic, 2011; Agbola, 2011; Kellough,
2002; Randell, 1994; Prowse & Prowse, 2009), b) the role of stakeholders in the
process (Andrews, Boyne, Meier, O’Toole, & Walker; 2012; Ayers, 2013; Kellough,
2002; Meier & O´Toole 2002, 2003, 2013; Kim 2011), and c) the cultural and
communication difficulties inherent to the evaluation process (Prowse & Prowse,
2009). As a matter of fact, these aspects have also been identified as crucial
factors explaining why performance assessments programs might fail in different
A. Performance Management and Organizational Strategy Legitimacy
As previously stated, the design of the performance management systems,
and the way they are translated into measurement instruments, can be also
convenient to explain the gap between the expected organizational results and the
actual ones. In fact, those formal standards for measurement can also help to align
the institutional arrangements, and the adjacent incentive system, to eventually
elicit greater organizational effectiveness (Buller & McEvoy, 2012; Kim, 2011).
Thus, the performance management system gives acceptability not only to the
performance measurement itself but also to the way the stakeholders adhere to the
organizational mission and strategy.
In other words, the degree of acceptance of the measurement system is
crucial to bring legitimacy to the whole management process. The evaluation policy
and the type of assessment managed, are expected to reflect the values and
expectations of the organization as a whole, and should convey a sense of fairness
and due process. Hence, so as to build reliability to the performance system, and
to improve the adherence to the whole organization strategy, the system of rules
must be straightforward, coherent and be perceived clearly by all the employees
(Agbola, 2011). When the performance management system is not well defined
and does not ease awareness by the employees, the process could have low
legitimacy and the instruments are not going to be perceived as strategic and
useful by individuals and groups within the organization (Delic & Cebic, 2011). To
sum up, it does not suffice to have a Performance Management System. For that
11
system of incentives for greater performance. More importantly, the individuals
must understand it, accept it and use it in their everyday work.
B. Appropriate Instruments of Measurement
An appropriate Performance Management System needs objective and solid
instruments, suitable to inform about the current state of the organization. The
performance assessments have a significant role because they give relevant
information to the decision makers about the organization status (Kim, 2011) and
its evolution. The resultant information does not only inform and generate
incentives to greater performance, but in due course, it provides reliability to other
organizational processes such as planning, decision-making and the definition of
the expected results.
To achieve those expected results, those assessments must be considered
valid, reliable, practical and fair by all the organization (Delic &Cebic, 2011).
Additionally, to become part of the organizational day-to-day, and to influence other
organizational practices, Kellough, (2002) and Agbola (2011) argue that the
performance assessments have to be a continuous process, more than a moment
in the year. For the authors, it is important to adequately implement the
performance measurement but, more importantly, to have a constant tracking,
feedback and revision in manifold directions (evaluator to employee, employee to
evaluator, and the like). This can allow early corrections in the process, and in the
performance of the employee and the organization, but also can help to improve
legitimacy. In conclusion, those mentioned features could help guarantee that the
to positively affect organizational processes and their legitimacy among
stakeholders.
Some authors have also found systematic weaknesses in the performance
assessments (Delic & Cebic, 2011; Kim 2011; Kellough, 2002; Murphy &
Cleveland, 1995), particularly related to methods and protocols. An important
source of bias and deficiency in the performance assessments is the rater error.
These errors, inherent to the human subjectivity or to each rater, introduce the idea
of bias, unfairness, and injustice among the employees enhancing dissatisfaction
with all the process (Murphy & Cleveland, 1995). These notions can be intensified
if the organizations do not have an appropriate rating scale. Not all the
organizations could be measured in the same way, but also, not all the employees
could have the same rating criteria. In this regard, the rating scales need to be
adjusted to the employee functions and to his outcomes, but also they have to be
able to reduce the rater biases at the same time (Kellough, 2002; Kim, 2011). The
systematic weaknesses in the performance assessments introduce noise and
doubts to the performance management process reducing the results reliability
among the organization community.
This is why the method of assessment, and the key processes attached to it,
(for instance how the information is recollected and conveyed) is essential to build
legitimacy throughout the entire performance framework. Any weakness as to how
those performance measurement processes and instruments are designed and
executed, can jeopardize the way the organization and its members react to them.
Those mentioned flaws threat the reliability of the system but more importantly,
13
the process can have overarching effects in the organization’s strategy and, hence,
it makes sense to inquire about the legitimacy of the performance assessment
process in public sector organizations, considering it social role and impact.
C. Goal alignment, managerial role and legitimacy
One of the key outcomes of a proper performance management system is
the alignment between the organization’s goals and the employee’s objectives. A
proper alignment would be ultimately reflected in group and employee behaviors,
which can positively affect the entire management outcomes. In fact, such
alignment is paramount to improve the organizational performance (Andrews et al,
2012; Kim, 2010) through bettering personal motivation and establishing a clear
feedback channel with the employee. Thus, another characteristic of a proper
performance management system is to establish a set of rules and incentives,
which in the end can lead to more clear communication between the employee and
the organization.
As a matter of fact, when goals and incentives are misaligned we could face
a classic agency dilemma, which is costly for the organization’s management. For
it induces significant distortions in the organization’s strategy and in the
organizational operation itself. An evaluation system that misaligns the incentives
and goals between the organization (the principal), and the employee (the agent),
can make more difficult the achievement of key organizational outcomes. Thus, a
proper evaluation system can help reduce transaction costs and strengthens the
psychological contract between the employee and the organization, increasing
There is another key role to be considered here, the managerial one. The
managers’ strategies can make difference in the result and can improve the
competitive advantage of the firm. (Buller & McEvoy, 2012; Meier y O´Toole,
2002). Moreover, managers can have a profound impact on the way goals and
incentives can be more easily aligned between the employee and the organization.
Meier and O’Toole (2002) found a positive relationship between the managers’
quality and organizational and individual performance. When top management
commits to the performance management process they introduce legitimacy to all
the activities that conform it (Kim, 2011). When managers internalize and
communicate the benefits of adhering to a performance management framework,
they present a clear message to all the employees about the importance to the
organization of the individual performance (Delic & Cebic, 2011; Kellough, 2002).
They can also help to refine the system and minimize error and biases (Ayers,
2013; Kellough, 2002). Our knowledge of the misalignment in objectives in the
Colombian evaluation system in the public sector and the role of management in
the process indicate that this scholarship is useful to understand the role of
legitimacy within the Colombian performance management system for the public
sector.
D. Organizational and Cultural Context and the legitimacy of the performance framework
Providing feedback is one of the fundamental procedures within the whole
performance management process (Kellough, 2002). For Delic & Cebic (2011) and
15
how to improve his performance and achieve the expected results, but to align
incentives and facilitate adherence to the whole organizational strategy. The
feedback process can also help to identify also the high (and low) performers and
help them to develop their potential (Hollenbeck et al., 2012). Thus, within this
process clear communication and responsiveness are essential to facilitate the
approach to the employee.
Recent scholarship has subsequently attempted to provide greater
importance to the social context that surrounds performance management
systems, in order to better understand its effects on organizational adherence and
legitimacy (Levy & Williams 2004, O´Toole & Meier 2014). For instance, Levy &
Williams (2004) stated that in order to understand and develop effective
performance appraisal, is necessary to identify, measure and define the
organizational context (2004:883). This affirmation is consistent with the Murphy
and Cleveland (1995) model whereby the context can be determinant of the rater
error and the employee performance orientation. However, this is a topic that still
shows little empirical research, for there is not a systematic empirical study in this
area and the results have reduced external validity (O´Toole & Meier, 2014).
However, these social context variables have to be taken into account when
attempting to explain the role of legitimacy within performance management
system and its effects on organizational performance.
Thus, to sum up, legitimacy plays a key role in ensuring that a performance
management system goes beyond its usual role of measurement and becomes a
key part of the organization strategy. The literature shows those different factors
managerial roles and bring more fluid communication within the organization that
leads to greater performance. The design and understanding of the system as a
whole, the design and operation of measurement instruments, the roles of the
involved actors, and the cultural and communication context, affect the way the
employees and the organizations perceive the performance management. When
the employees do not perceive the performance appraisals as an effective, and
fair, way to achieve personal growth and professional development, the whole
performance framework is likely to fail, but, more importantly, underperformance
can emerge (Kim, 2010). Our effort in this article is to explore the role that
legitimacy plays on the effectiveness of the performance management system of a
country with civil service still in development. Prior literature will help us to
understand the role that the lack of legitimacy can bring, not only to the
performance management framework, but also to the organization performance as
a whole.
II. CONTEXT OF THE STUDY
Colombia is a unitary decentralized country in the north of South America.
This country adopted merit in civil service as a principle of the 1991 political
constitution. From that moment onwards, the government has strived to achieve
merit-based procedures on public employment, including the adoption of a
performance management system. Successive governments have introduced a
series of institutional arrangements aiming to adopt a comprehensive management
17
Most of the governmental efforts have aimed to facilitate the adoption and
enforcement of the performance management system as part of public sector
employment procedures. A law issued in 2004 defined the appraisal´s scope and
made evaluations mandatory at all levels of government and agencies. The law
provided a governance structure for the system and identified the key actors and
their responsibilities’ within the evaluation process. Accordingly, the National Civil
Service Commission (NCSC), the organization that rules and controls the
Colombian public service, has continuously been specifying and building the
performance assessment system according to the general indications of the
Ibero-American Charter of Public Management (CLAD, 2003)2. The NCSC has also
defined the general standards of performance for the system’s implementation,
allowing some leeway to both organizations and subnational levels of government
to adapt the evaluation instruments to their own particularities. Thus, it can be
possible to argue that there exists already a general (de jure) framework in
Colombia’s public sector to adopt performance management as a strategy within
public organizations, however, as is usual in performance management systems
the main challenges appear during the implementation process.
Throughout the implementation of the performance assessment process, there
are two key actors: the NCSC and the public agencies. Besides defining general
standards for the performance evaluation system, the National Civil Service
Commission has created the standard instruments and guidelines for
implementation, and must ensure their adoption by Colombian public
2 The Ibero-american Charter of Public Management is a non-binding common framework implemented since 2003, which
establishes the public function bases to improve the government performance and strengthen democracy. The Charter principles must be adapted to the countries’ context by a normative development.
organizations. On the other hand, the agencies are compelled to implement the
national policy in response to the most recent normative efforts. However, even
though the system allows organizational and regional adaptation, as of this
moment only eighteen (18) organizations (out roughly 6,500 in Colombia) have
developed their own appraisal systems. The implementation of the performance
management process operates as a collaborative process between the central
government and each public organization. However, different limitations in both
formulation and implementation have diminished its impact as an organizational
tool.
According to the law, there are three types of performance assessments in
Colombia’s public sector. The first type is probationary period evaluation, the
second ordinal, and the third, extraordinary. The probationary period evaluation
aims to assess the employees’ principal activities and responsibilities in the
six-month probationary period. The second and third assessments types are
specifically addressed to career employees. The ordinal evaluation is made of two
semi-annual evaluations and a final year round evaluation. Additionally, the
extraordinary assessment allows an extra evaluation of the employee when the
superior has evidence of unsatisfactory performance after the last performance
assessment. Yet, all the evaluation types are part of the annually based
performance assessment process.
According to the normative framework, in Colombia the performance evaluation
process is continuous and has five main components (see figure 1). For the
evaluation to be conducted, the organization has first to create or update the job
19
manager and the employee. After the goals and commitments have been
established, the manager and the employee decide what is going to be measured.
The supervisor has to track the officer’s performance and creates an evidence
portfolio. With this information, the superior develops the semiannual evaluation,
which is a partial assessment that will help to improve the officer performance
through providing her feedback by the middle of the year period. After this moment,
the monitoring process continues until February when the final evaluation takes
place.
Figure 1. Performance Assessment Process3
It is important to highlight that, according to the law, performance
assessments are conceived as a key tool of Human Capital Management (HCM).
De jure, they are considered binding for HCM processes such as promotion,
training and professional development. Yet, even though the law states that the
appraisal results must be used to make decisions about tenure, and other key
employee outcomes, there is a gap between the expected and the actual results of
performance assessments. Despite the Colombian government efforts to
implement the performance system, according to the OECD (2013), it does not yet
operate as a proper and effective performance management framework.
3 Acuerdo 137 de 2010
Jan.
Feb.
Mar.
Apr.
May
. Jun. Jul. Aug. Sept. Oct. Nov.. Dec. Jan. Feb.
Mid-‐period assessment (August 31)
Final Assessment
(Feb 15)
Fixed Commitments
(January 31)
Monitoring and conformation of the portfolio of evidences (Feb 1 – Jul 31)
Monitoring and conformation of the portfolio of evidences (Aug 1– Jan 31)
Nowadays, despite the negative perceptions from the citizenry about the quality of
public services, a significant proportion of the staff consistently receives high
performance ratings. In this context, low performance is widespread over the
government and the managers find quite difficult to use the system as a tool to
motivate public servants (OECD, 2013: 336). The limitations of the implementation
reduce the general legitimacy of the assessment among public officials and
supervisors, leading to a vicious circle, which does not help to improve either the
assessments or the actual organizational performance.
Recent scholarship on the Colombian process illustrates the shortcomings
of the evaluation process in Colombian organizations and makes caveats about
their effects on legitimacy. There is a widespread recognition that the performance
evaluations are ill defined and adopted (Careaga, 2013, Ospina & Hoffman, 2015).
According to interviewed human resource officers, the appraisal is not an effective
tool and is difficult to connect with the HCM strategy (Careaga, 2013). The HCM
officers perceive a wide disconnection between formulators and implementers at
both the macro (national) and the micro (subnational and organizational) levels
(Ospina, 2000; Ospina & Hoffman, 2015). Furthermore, the evaluation system
receives little recognition from key actors involved in the process as a tool for
organizational and individual improvement (Strazza, 2014:29), and it does not help
to generate a proper incentives framework, reducing the evaluators’ commitment to
the process (Sanabria et al, 2015). Those flaws hinder the advancement of the
system as a management tool and, more importantly, permanently erode the
legitimacy of the Governmental Performance Management System among both
21
about the legitimacy challenges that civil services still in development can face but,
more importantly, it can also illustrate how those legitimacy problems relate to the
organization’s performance.
Some actions and norms that currently regulate the performance
assessment system generate incentives that jeopardize it legitimacy. For example,
the career system implemented and ruled by the 909 law is more a position system
and do not allow internal promotion. In this sense, the assessments are tied with
the planning process, capacitation and temporary commissions. This situation
could lead employees to give greater value to stability and they would not be willing
to do more effort in pursue of an excellent performance.
In the other hand, the current regulatory framework places the responsibility of a
job reinstatement in the evaluators. In these situations, evaluators they must
respond with their financial assets. The above situation can encourage evaluators
to remain outside of the process and to prefer to evaluate with good results instead
of having the risk that regulation imposes. Consequently, some perverse situations
arise in the current system in order to ensure competent management teams.
III. HYPOTHESES, DATA AND METHODOLOGY
As we mentioned before, our research question is focused on the determinants
of the perceptions of legitimacy of performance assessments among Colombian
public officials. In order to answer this question we established three main
hypotheses according with the literature review and we tested them with a
qualitative and quantitative approach. In this section, we bring a detailed
A. Research Hypotheses.
In order to answer the research question, we established three main
hypotheses according to the literature review. These hypotheses are based on the
relationship between key determinants of the performance assessments and their
legitimacy among public employees.
First, Buller and McEvoy (2012) and Meier and O’Toole (2002) demonstrate
that managers’ strategies and the way they approach to problems can make the
difference in final results and may improve the competitive advantage of the firms.
Furthermore, the managers’ attitude and the managerial tools to develop
teamwork, build knowledge, and make processes of double-loop learning, can be
related with the way these managers use and consider the performance
assessments. The way the officers perceive the managers role is related with the
cognitive legitimacy concept because it observes if the activities developments are
the most appropriate (Suchman, 1995). It is also involved in the legitimacy
construction process developed by Fambry and Harper (2005). Therefore, our first
hypothesis is:
H1: The relationship between the managers’ role and the legitimacy of
performance assessments among the public employees will be positive.
The alignment between the individual and organizational goals and values is
fundamental to achieve desirable outcomes. When goals and incentives are
misaligned, the organization could face an agency dilemma, which implies high
costs to the firm (Andrews et al, 2012). In this way, the organizations must try to
23
behaviors, which can positively affect the entire management outcomes (Kim,
2010). Additionally, the performance management system must be able to align the
organizational strategy and the system of incentives for greater performance. The
individuals must understand the performance management system, accept it and
use it in their everyday work. In this sense, the performance assessments must be
able to align the individual performance with the organizational goals and outputs
in order to be legitimate to all the employees. Consequently, our second
hypothesis is:
H2: The relationship between the individual and organizational assessment
alignment and the legitimacy of performance assessments among the public
employees will be positive.
Finally, the perception about key organizational processes can reflect the
employees’ attitude about the rest of processes. Hence, when the procedures have
systematic weaknesses, it introduces noises and doubts, reducing the results
reliability among the organization community. The transparency perception is
important to consider also the cognitive legitimacy because it is focused on
identifying how processes are performed and if these correspond to the criteria
previously established as appropriate. In this sense, our last hypothesis is:
H3: The relationship between the transparency perception and the legitimacy of
performance assessments among the public employees will be positive.
B. Data and Methodology.
As mentioned previously, in this study we use both a qualitative and
analysis to approach the performance assessments practices in the regional
entities. We used a semi-structural interview to explore the performance
assessment implementation in four subnational government organizations (annex 1
– Interviews Format in Spanish). Our qualitative analysis focused on the human
resources managers’ perception about the performance assessment and its
implementation within their organizations, keeping in mind the key determinants
that were listed before. The interviews took place in different subnational
governments with diverse administrative, economic and social conditions. Table 1
contains information about the selected subnational governments and its
quantitative models control variables.
Table 1 Interviews Descriptive Data
Province Antioquia San Andrés Chocó Valle
Organization Gobernación de Antioquía
Gobernación de San Andrés, Providencia y Santa Catalina.
Alcaldía de
Quibdó Alcaldía de Cali
Type Gobernación Gobernación Alcaldía Alcaldía
Capital City Medellín San Andrés Quibó Cali
Municipal Size
Category Especial 5 6 Especial
Provincial Yearly
GDP (2012)4 63.805 680 1961 46.289
Provincial Yearly GDP Growth
(2011-2012) 4% 3,9% -3,8% 3,8%
Distance 251,9 km 1217,9 km 309,3 km 292,9 km
Abstention 46,6% 45,3% 39,1% 49,1%
Sources: Resolution 705 de 2013 Contaduría General de la Nación, Provinces National Accounts, Google Earth and Registraduría Nacional del Estado Civil.
4 Billions of pesos
25
The interviews were transcribed and codified using NVIVO software (Annex
2 - coding dictionary). This software allows us to identify the principal ideas and link
the quantitative results with the theory framework.
From the above, we continued with a quantitative analysis of the legitimacy
of performance evaluations. Here we take the public employees (individuals) at the
province level (departamentos5) as our unit of analysis. The main source of the
information for our independent variables is the Survey of Institutional Environment
and Development (EDID by its acronym in Spanish) conducted by the Colombian
Statistics Agency (NSD). The EDID survey collects information about public
employees’ views on several aspects of working environment and organizational
performance (NSD, 2013). Conducted on a yearly basis, we examine data between
2009 and 2012, which implies a sample of 19.000 observations6. As our control
variables, we use organizational and subnational characteristics in order to capture
the differences between the subnational public employees. We introduce variables
as the Administrative Capacity Index (ACI) developed by the National Planning
Department (NPD). Such index provides information about the administrative
capacity of the subnational governments and their effectiveness at different levels.
Although the EDID is a perception-based survey and the ACI is partially based on
self-reported information, the size of the sample helps to control for common
source bias and other measurement errors usual in such type of data. Table 2
listed our dependent and independent variables:
5 Colombia is divided in 32 provinces (departamentos in Spanish) and the capital city Bogotá
6 Additionally we use other control variables like the provinces´ GDP, the distance between the province capital and Bogotá,
Table 2 Variables Description
Variable Answer Description EDID Questions Source
Legitimacy Incentive (First Dependent variable)
Ordinal Categorical
Public officer views as to how the performance appraisal operated to improve her/his performance
- The implementation of officials’ performance
evaluation encouraged
officials to improve their work.
Survey of Institutional
Environment and
Development
Colombia's Statistics Office (NSD)
Legitimacy Input (Second
Dependent Variable)
Ordinal Categorical
Public officer views as to how the performance appraisal operated as an input of the organization performance management system.
- The implementation of officials’ performance evaluation was taken into account to improve the service provided by the entity.
Survey of Institutional
Environment and
Development
Colombia's Statistics Office (NSD)
Managerial Role (Independent variable)
Index (0 to 100)
Index built with perception questions on the managers legitimacy (team building, direct knowledge, double-loop learning)
-The Directors had full knowledge of the difficulties and solutions presented in the development of the tasks.
-The Directors took into account the contributions of the teams in the decision-making process.
-The Directors led to personal and technical growth of its team.
Survey of Institutional
Environment and
Development (NSD)
Individual and organizational assessment alignment (Independent variable) Ordinal Categorical
Use of individual performance assessment in the evaluation of the global government performance
- For the institutional evaluation: How much has considered the result of performance evaluation of officials?
Survey of Institutional
Environment and
Development (NSD)
Transparency perception (Independent variable)
Index (0 to 100)
Index constructed with perception questions about transparency on key organizational processes (Budgeting planning and allocation).
Programming and budget execution is based on:
-Personal, family or
friendship ties?
-Ties or political pressure? -Unofficial payments? -Logrolling?
-Regional links?
Survey of Institutional
Environment and
Development (NSD)
CONTROLS
Variable Answer Description Source
Provincial Yearly GDP Billions of Colombian pesos Provincial yearly GDP at 2005 constant prices. Provinces National Accounts Provincial Yearly GDP
Growth Percentage Provincial yearly GDP growth between 2011 and 2012 at 2005 constant prices. Provinces National Accounts Municipal Size Category Ordinal Categorical Provinces according to the population size capital classification Resolution 705 de 2013 Contaduría General de la Nación
27
CONTROLS
Variable Answer Description Source
Administrative Capacity Index (0 to 100) Index of Municipal Performance (IMP) Integral Municipal Performance National Planning Department (NPD)
Abstention Percentage Percentage of the eligible voters who did not vote in 2011 local elections Registraduría Nacional del Estado Civil
In this study, the first dependent variable (Legitimacy Incentive) refers to the
employees’ perception about the assessments capacity to induce a positive
change in the public official's behavior. It is an ordinal categorical variable, which
takes values between one and five. Higher values in the variable represent a more
positive perception of effectiveness in improving performance. It does not only
measure how legitimate they are perceived by public servants, but also how useful
they consider them to improve organizational performance. For this variable, we
rely on 19,471 observations between 2009 and 2012 with a mean of 3.28 and 1.23
as standard deviation. The subset distribution is left skewed. However, almost 40%
of the employees do not perceive the performance assessments capable to induce
a positive change in the public officers` behavior.
According to Gerrish (2015:15), allowing the public officers to know the
performance measurement results may be associated to better performance
conditions. In this sense, the second independent variable is Legitimacy Input. It
refers to the public servants perception about the use of assessments' results as
an information input to improve the organization's performance. We have 19.471
observations of this variable between 2009 and 2012 with a mean of 3.56 (1.2
standard deviations). This variable has a left skewed distribution and more than
70% of the subset considers the assessments as an input in the performance
The first independent variable of interest is Managerial Role. It measures
the employees’ perception about the manager´s attitude and the managerial tools
at hand. Manager's role and attitude can have a positive effect in the organizational
and individual performance (Kellough, 2002; Kim, 2010; Meier &O´Toole, 2002,
2003; Prowse & Prowse, 2009). This index ranks between 0 and 100, and has a
77.4% of internal consistency (Cronbach's Alpha). The mean is 65, which
indicates that the perception about the managers’ role in the provincial
governments is over the median value. The standard deviation of this variable is
23.1
The second independent variable is Individual and Organizational
Assessment Alignment. This variable measures whether the performance
assessments are able to align the individual performance with the organizational
goals and outputs. (Ayers, 2013; Kellough, 2002; Kim, 2001; Murphy & Cleveland,
1995). This is an ordinal categorical variable that reflects the employees’
perceptions as to how the individual performance assessments are taken into
account within the global organizational performance evaluation. This variable
distribution is left skewed with a mean of 3.6 and 1.3 standard deviation, which
imply a positive employees’ perception about the alignment between the
organizational performance evaluation and the individual performance
assessments.
Finally, the Transparency Perception variable measures the employees’
opinions about how well the budget is planned, allocated and executed. Higher
values in this index suggest better perceptions about budgeting processes in the
29
Cronbach’s Alpha. In this case, the subset mean is 49.3 (with 28.3 standard
deviation), which implies that the perceptions about transparency in managerial
practices are neither positive nor negative.
To capture factors that explain the organizational and regional heterogeneity
we added some control variables. The economic productive differences are
controlled with the provincial yearly GDP (2005 constant prices). The second
control is the percentage of the provincial yearly GDP growth between 2011-2012.
Additionally, differences in population sizes are measured with the Colombian
municipal classification system 7 . The third control variable measures the
separation between national and subnational government in public policy
implementation. The administrative capacity variable measures different aspects of
municipal activity, as well as the local governments' capacity (human and
technologic resources availability management abilities) (DNP, 2012). Finally,
voting abstention measures the percentage of the eligible voters who did not vote
in 2011 local elections, capturing the local perception and trust about the local
governments.
Table 3 Descriptive Statistics.
Variable N Mean SD Min Max
Legitimacy Incentive (First Dependent
variable) 19,471 3.3 1.2 1 5
Legitimacy Input (Second Dependent
Variable) 19,471 3.6 1.1 1 5
Managerial Role
(Independent variable) 19,471 65 23.1 0 100
7 Nowadays, Colombia has 1.098 municipalities that are classified in seven categories according to population and income levels (artículos 1 y 2, Ley 617 del 2000). In 2007, 89% of the municipalities were classified in the lowest category (six), and only five cities were considered in the special category (Federación Nacional de Municipios, 2013).
Variable N Mean SD Min Max Individual and organizational
assessment alignment (Independent
variable) 19,471 3.6 1.3 1 5
Transparency perception
(Independent variable) 19,471 49.3 28.4 0 100
Provincial Yearly GDP 19,471 35,798.3 43,321.3 118 122,685
Provincial Yearly GDP Growth 19,471 0.04 0.03 -5.1% 11.9%
Municipal Size Category 19,471 1.8 0,9 1 4
Distance 19,471 299.6 287 0 1217.9
Administrative Capacity 19,471 77.1 15.2 0 97.9
Abstention 19,471 42.3 7.6 23.8% 52.6%
In this study we also consider some variables in order to control the latent
heterogeneity of the Colombian context. First, we include variables to introduce the
sub national economic differences into the analysis. The variables Provincial
Yearly GDP and the Provincial Yearly GDP Growth represent a proxy for economic
development of the departments during the study period. Both variables describe a
profoundly different behavior between the provincial governments (see table 3), but
they present a biased behavior of the sample to the left. In this context, some
special cases modify the aggregate results.
Taking into account the differences in the capacity of the local governments
we include two variables: Municipal Size Category and Administrative Capacity. In
the first case, the variable reflects the classification of the province capitals
according to their population and resource management. On the other hand,
administrative capacity is a variable that aims a measuring processes related to
local government areas conducted by the NPD. Both variables capture the existing
heterogeneity in these areas and allow comparisons regarding the determinants of
31
Finally, we introduce two other variables to control aspects such as
decentralization and the legitimacy of governments. It was used a proxy for the
distance between Bogotá and the provincial capital. On the other hand, the level of
abstention in the last election seeks to capture the legitimacy of governments. To
incorporate these variables allows differences as the remoteness of the formulator
organ of political and electoral climate no influence on the outcome of interest in
this work.
To answer our research question we estimated two ordinal probit models
considering that our dependent variables are categorical with more than two
options (Long & Freese, 2001). According with the variable description, the models
are:
Model 1:
𝑃𝑟(𝑙𝑒𝑔𝑖𝑡𝑖𝑚𝑎𝑐𝑦_𝐼𝑛𝑐𝑒𝑛𝑡𝑖𝑣𝑒! =𝑚|𝑥!)
= 𝜙 ß!+ß!𝑚𝑎𝑛𝑎𝑔𝑒𝑟𝑠_𝑙𝑒𝑔𝑖𝑡𝑖𝑚𝑎𝑐𝑦! +ß!𝑎𝑙𝑖𝑔𝑚𝑒𝑛𝑡!
+ß!𝑡𝑟𝑎𝑛𝑠𝑝𝑎𝑟𝑒𝑛𝑐𝑦_𝑝𝑒𝑟𝑐𝑒𝑝𝑡𝑖𝑜𝑛! +ß!𝐺𝐷𝑃! +ß!𝐺𝐷𝑃_𝐺𝑟𝑜𝑤𝑡ℎ!
+ß!𝑚𝑢𝑛𝑖𝑐𝑖𝑝𝑎𝑙_𝑠𝑖𝑧𝑒_𝑐𝑎𝑡𝑒𝑔𝑜𝑟𝑦! +ß!𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒!
+ß!𝑎𝑑𝑚𝑖𝑛𝑖𝑠𝑡𝑟𝑎𝑡𝑖𝑣𝑒_𝑐𝑎𝑝𝑎𝑐𝑖𝑡𝑦!+ß!𝑎𝑏𝑠𝑡𝑒𝑛𝑡𝑖𝑜𝑛! +𝜐!
Model 2:
𝑃𝑟(𝑙𝑒𝑔𝑖𝑡𝑖𝑚𝑎𝑐𝑦_𝐼𝑛𝑝𝑢𝑡! = 𝑚|𝑥!)
= 𝜙 ß!+ß!𝑚𝑎𝑛𝑎𝑔𝑒𝑟𝑠_𝑙𝑒𝑔𝑖𝑡𝑖𝑚𝑎𝑐𝑦! +ß!𝑎𝑙𝑖𝑔𝑚𝑒𝑛𝑡!
+ß!𝑡𝑟𝑎𝑛𝑠𝑝𝑎𝑟𝑒𝑛𝑐𝑦_𝑝𝑒𝑟𝑐𝑒𝑝𝑡𝑖𝑜𝑛! +ß!𝐺𝐷𝑃! +ß!𝐺𝐷𝑃_𝐺𝑟𝑜𝑤𝑡ℎ!
+ß!𝑚𝑢𝑛𝑖𝑐𝑖𝑝𝑎𝑙_𝑠𝑖𝑧𝑒_𝑐𝑎𝑡𝑒𝑔𝑜𝑟𝑦! +ß!𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒!
+ß!𝑎𝑑𝑚𝑖𝑛𝑖𝑠𝑡𝑟𝑎𝑡𝑖𝑣𝑒_𝑐𝑎𝑝𝑎𝑐𝑖𝑡𝑦!+ß!𝑎𝑏𝑠𝑡𝑒𝑛𝑡𝑖𝑜𝑛! +𝜐!
Our dependent variables are defined as:
𝑙𝑒𝑔𝑖𝑡𝑖𝑚𝑎𝑐𝑦! =
1 𝑖𝑓 𝑖𝑠 𝑡𝑜𝑡𝑎𝑙𝑙𝑦 𝑑𝑖𝑠𝑎𝑔𝑟𝑒𝑒
2 𝑖𝑓 𝑖𝑠 𝑑𝑖𝑠𝑎𝑔𝑟𝑒𝑒
4 𝑖𝑓 𝑖𝑠 𝑎𝑔𝑟𝑒𝑒
Where the employee could have different levels of agreement (totally
disagree, disagree, agree and totally agree) about the role of the performance
assessments as a management tool in the mentioned two dimensions (incentive
and input).
IV. RESULTS
A. QUALITATIVE RESULTS
In this study, we conducted an interview8 analysis according with a
deductive approach where, according with the literature review, we tried to prove
hypothesis based on the quantitative results. First, we observe the interviews word
frequency in a cloud graph, built with NVIVO software, where the most repeated
words have the biggest size. Figure 2 presents the results in Spanish of all
interviews conducted in this work. We observe that words like assessment,
performance and system have a central role in this graph. It is followed by words
like process, commitments, manager, career, officers, plan, government,
evaluated, evaluator, etc. Annex 3 contains this graph for each city where were
conducted the interviews.
Another way to analyze the results is exploring the common points and
trends found in the statements. We introduce fragments of the interviews in
Spanish to strengthen the analysis where is not possible to identify who expressed
the statement.
8 These interviews were made within a research project about Human Resources Management in Colombian public sector
organizations. This project is implemented by the Universidad de los Andes, funded by the Higher School of Public Administration (ESAP by its Spanish acronym), with the support of the Administrative Department of Civil Service (DAFP by its Acronym in Spanish) and the Administrative Department for Science and Technology of Colombia (COLCIENCIAS).
33 Figure 2. World Frequency Cloud Graph
Source: Authors analysis with NVIVO10 Software
In general, we can observe that management role, management tool, and
alignment are the principal concepts the interviewers mentioned. On the other
hand, the concept transparency perception was not prevalent in human resources
managers’ statements.
The managerial role was the determinant that was mentioned the most.
Some of the human resources managers’ statements were:
“…es más desde el empoderamiento por parte de cada jefe de dependencia de asimilarlo como una muy buena práctica de gestión dentro de la dependencia. A veces encontramos que se toma más como un instrumento que permite generar mayor carga y no beneficio. O sea lo toman más como algo que hay que hacer. Pero no lo toman como un sistema que le va permitir a las dependencias mejorar.”
“prácticamente que el evaluador no tienen conocimiento exacto de lo que es el plan de gobierno, el programa de gobierno. Las metas institucionales. Realmente lo que es. Las metas de la institución. Entonces de pronto se conciertan los compromisos por fuera de eso. A lo que de
pronto estima conveniente o máximo con lo que dice el manual de funciones. No más. Es sobre eso.”
These two quotes illustrate the managers’ role importance in the process. In
addition, it shows how the human resources managers link up the managers’
attitudes with the performance assessments implementation.
Management tool, managers’ role, and legitimacy appear in all interviews,
what leads to think about the key role of these concepts in the performance
assessment process. On the contrary, in San Andres, Antioquia, and Cali, the
interviewed made recommendations to improve the current system and describe
developments that have been implemented by the subnational governments in
order to strengthen the performance assessment process. Only in Quibdó was
mentioned the transparency perception as a key component of the performance
assessment system.
In all statements where we identified the legitimacy and the transparency
perception concepts, the human resources managers conveyed a negative
perception. In few cases, the human resources managers express positive
testimonies about Managerial Role and Alignment-coded sections of transcriptions.
Some of these testimonies were:
“Entonces recae mucho la responsabilidad sobre ellos y en los últimos tres años, ellos han sido un poco más sensibles, han estado escuchado un poco más y se ha respondido de tal manera que ha aumentado mucho, por lo menos la tarea. Hacer la evaluación. Ya que la estén utilizando como una herramienta de gestión, hasta allá no estamos llegando todavía. Pero si por lo menos se están dando cuenta que hay que comenzar a evaluar”.
“Porque ellos se han interesado en que se evalúe. Lo que de pronto no se han sentado es a concertar bien. Todavía tenemos que los evaluados tienen que participar como mucho, mire ya
35 es hora que me evalúe, mire que ya es hora que me concerte. Si los jefes tienen que hacerlo pero más como un requisito, más que como una herramienta que les esté sirviendo. Si se ha logrado pues que se mire la evaluación, pero necesitamos avanzar”.
“Todo desempeño laboral debe ser evaluado. Entonces desde ahí es que van inmersos los planes de mejoramiento individual, la valoración de las dependencias como van, si están cumpliendo las metas del plan de desarrollo, si sí se están llevando a cabo los procedimientos o procesos atados a cada dependencia. Entonces como tal el sistema si es pertinente para la administración implementarlo”.
The interviewed human resources managers are aware of the determinant
role of managers in the performance assessments process. In addition, they
emphasize in how alignment between the individual and organizational strategy
and goal is important. In the Colombian system, the alignment begins with the
commitments definition in February.
In a very predictable way, positive affirmations were prevailing in sections
about improvement and managerial tool. Some of the testimonies from human
resources managers regarding this topic were:
“En qué sentido, en que definitivamente todos los servidores públicos, independiente del nivel jerárquico que se maneje, tiene que fijársele unos compromisos al inicio, hacer un seguimiento y una retroalimentación a esos compromisos y una evaluación definitiva … Entonces como tal, de entrada, que todo servicio profesional o laboral tiene que ser evaluado en el tiempo. Entonces desde ahí el instrumento como tal le ha posibilitado a cada jefe o superior de área hacer el respectivo seguimiento que amerita el desempeño”.
“Formalmente, la evaluación de desempeño sirve para mirar la mejora de competencias de los servidores públicos, para establecer los compromisos, para evaluar el desempeño de los servidores, si formalmente. Y sirve para, desde la perspectiva de las competencias que se deben mejorar, entonces fijar los propósitos del Plan Institucional de Capacitación. Y también sirve para el programa de estímulos y bienestar de las entidades públicas. Entonces, yo con una evaluación del
