ESTABLECIMIENTO: H-III CORONEL DU GRATY

Evaluation and research on the effectiveness of leadership development programs outside of the library sector, while more prevalent, has still been lacking. Evaluations have been done infrequently, and results of leadership development interventions have not been reported (Romaniuk & Haycock, 2011). When evaluations have been done they typically have measured the interpersonal skills and work performance of participants and have not considered the effect of the program on the larger community (Black & Earnest, 2009; Russon & Reinelt, 2004). Those programs that assessed outcomes related to personal development seldom assessed the effectiveness in achieving outcomes related to improving organizational performance (Collins, 2002; Lai, 1996). Owen (2008) suggested that

evaluation of college student leadership programs have suffered from the same limitations as in other sectors and have been limited to single institution studies, suggestions for program content unrelated to program outcomes, anecdotal evidence and generally weak evaluation methodology.

Not unlike the library sector, many organizations have assumed that leadership

development efforts would improve organizational performance. Some research supported this belief. Leadership development level was a strong predictor of performance as evaluated by peers and subordinates (Strang & Kuhnert, 2009). However, improvements in

organizational performance were not necessarily guaranteed with enhanced knowledge and skills in employees (Collins & Holton, 2004).

Based on the results of a meta-analysis, leadership development program interventions had a significant effect on the knowledge and behavioural outcomes of the participants but

had a considerably lesser effect on the performance of the organizations to which the participants belonged (Collins, 2002; Collins & Holton, 2004). A meta-analysis, undertaken by Collins (2002), of 83 leadership training programs evaluations between 1982 and 2001 showed that there is a positive correlation between participation in a managerial leadership development program in the private sector and knowledge outcomes. The effect size for knowledge outcomes ranged from .96-1.37. There was a lesser effect for expertise or behavioural change outcomes ranging from .35-1.01 and only a small effect for

organizational or performance outcomes of .39. The findings of Collins, as well as Collins and Holton (2004) were summarized by Romaniuk and Haycock (2011) who indicated that after a leadership development participants were likely to have experienced a significant change in their knowledge and a change, albeit somewhat lesser, in their behaviour but the intervention had only a small effect on the organization which they were affiliated, as defined by organizational performance. This finding, albeit related to corporate sector programming, substantiated the continued existence and prevalence of leadership development programs and the continuing relevance of the research.

Research in other sectors had been precipitated by funding agencies which demanded accountability for their investment in leadership development programming. Spurred on by changes in the W.K. Kellogg Foundation, which had a long history of support for leadership development interventions, Russon and Reinelt (2004) were commissioned to undertake a study to provide a scan of current evaluation efforts for change oriented leadership programs. Not all of the 55 programs participating in the research were funded through the W.K.

Kellogg Foundation. However, the research findings mirrored those in the private sector. Romaniuk and Haycock (2011) reviewed the research that suggested that the evaluation of

leadership development programs has occurred but has typically been directed to personal outcomes such as knowledge, skills, attitudes, perceptions and changes in behaviour. There has been little evaluation relating to effects at the organizational or community level (Russon & Reinelt, 2004).

However, some findings in the not for profit and public sector research were not evident in the private sector based research. Few programs in the not for profit sector had any basis for the evaluation that was undertaken (Russon & Reinelt, 2004). That is, few programs were based on any specified theory as to how the program was to achieve the intended outcomes. Without a program theory it was difficult to know what to evaluate (Russon & Reinelt, 2004). Thus, a program might have intended to achieve results at the organizational level but undertook a program that was designed to change individuals. The subsequent evaluation did not measure the attainment of organizational results but rather whether the program achieved the individual development that the program was inappropriately designed to do (Russon & Reinelt, 2004). Dugan and Komives (2007) conducted a national evaluation of college student leadership development programs in the United States to determine what experiences derived desired outcomes. The work was significant because it represented the first attempt to examine student leadership development in the context of outcomes. However, the

outcomes were not specified or derived from the individual programs but rather based on the theoretical underpinnings of the research, The Social Change Model of Leadership

Development. Thus, the research provided a theory as to how the programs were to have achieved the intended outcomes, but as Russon and Reinelt (2004) observed, the programs themselves were generally devoid of the theory.

Evaluations conducted in the not for profit and public sector tended to be based on short term outputs such as number of participants or satisfaction rather than on longer term outcomes (Russon & Reinelt, 2004). This finding was consistent with the findings of Black and Earnest (2009), also in reference to programs funded through the W.K. Kellogg

Foundation, that few published studies measured the change that the participant experienced and then measured how that change affected the community or organization with whom the participant interacted. Black and Earnest undertook to develop a mechanism by which leadership development programs could be evaluated on the basis of outcomes. Ideally, leadership development programs should be measured on three levels of outcomes

individual, organizational and community (Black & Earnest, 2009). In testing the trilateral model for evaluation Black and Earnest determined that leadership self-efficacy was increased as a result of participation in an intervention.

These studies were indicative of the difficulty in assessing leadership development programs and in ascertaining meaningful results. The findings substantiated the fact that the lack of research and generalizable evaluation impaired knowledge about leadership

development. However, while several studies have been critical of the quality of program evaluation research, Avolio et al. (2009) analyzed 138 studies in the domain of leadership and found that experimental and quasi experimental studies of leadership interventions showed that the interventions had a positive affect across a broad array of intervention type, organization types, leadership levels, theories and outcomes such that leadership

interventions have made a positive difference. Thus, leadership development interventions have confirmed utility, but there has been little understanding as to why some programs have had the outcomes they have. As with library specific research, leadership self-efficacy

emerged as a by-product of the intervention. This suggests the relevance of this research to better understand LSE and its effect on leadership development.

Pluzdrak (2007) undertook one of the few studies that established a correlation between behavioural change as a result of a leadership development program and organizational improvements such as profits and turnover in the case investigated. Leadership competencies and behavioural changes in customer focus, interpersonal savvy and drive for results were positively correlated with organizational improvements while decision quality, ethics and values and integrity and trust were positively correlated to profit (Pluzdrak, 2007). The research was relevant as it correlated findings to meaningful organizational measures, and while not necessarily generalizable, it suggested methodologies that could be adapted for the evaluation of library LDP.

As in the library sector, LDP have resulted in positive changes to individual skills and behaviours. One of the most significant effects was the development of leadership self- efficacy (Augustin, 2003; Dugan & Komives, 2007; Feeney, 2006; Harris & Leberman, 2012) even though it was not an anticipated outcome of the program. Moreover, self-efficacy is one of the five essential components of a leadership development program (Olivares et al., 2007). This research was indicative of the importance of leadership self-efficacy

enhancement as a leadership development outcome.

The development of self-awareness has been an important outcome of LDP and has represented the area of greatest change in competency or knowledge in programs directed at senior executives (Fleishman, 2005; McCauley & Hughes-James, 1994; Pluzdrak, 2007) and with college students, defined as emerging leaders (Dugan & Komives, 2007). McCauley and Hughes-James (1994) and Fleishman (2005) conducted studies on leadership

development programs for school superintendents. While one of the most important

outcomes of a LDP for the superintendents related to self-awareness (53%), only 13% of the participants experienced an outcome related to self-confidence (McCauley & Hughes-James, 1994). This may suggest that leadership self-efficacy may not be an important dimension of leadership development for senior or experienced managers or leaders. Both studies noted that the superintendents, while often new to their role, were typically experienced educators, managers and even leaders. This appears to be consistent with the research of Pluzdrak (2007) who did not report any results for leadership self-efficacy in executives.

Other research, consistent with the self-efficacy construct proposed by Bandura, found that novice teachers, those with less experience, had lower self-efficacy relative to their teaching capabilities than did their more experienced counterparts (Tschannen-Moran & Hoy, 2007). The collective findings suggested that it was important to define the intended research in terms of emerging leaders as the effect of LDP on the leadership self-efficacy of executives was uncertain and the effect on senior managers appeared to have been quite different than reported for new leaders.

Several studies addressed appropriate LDP content and linked that content to the program outcomes. Leadership behaviours improved after taking part in action learning programs that incorporated reflection and questioning (Lee, 2005). Lee (2005) linked characteristics associated with visionary leadership to action learning and determined that vision, follower- centeredness, reward-equity, caring and confidence were affected, but communication was most affected. Findings relative to content, especially mentoring, were consistent with those in the library sector and other private sector research. Mentoring was an important

2006; Fleishman, 2005). The research provided some insight as to the attributes of programs that generated behavioural results and influenced leadership self-efficacy that has informed the research herein, particularly as it related to the qualitative data analysis.

While much of the research showed that specific leadership development programs had some positive effect on the participants or organizations, other research suggested that behavioural changes did not occur with participation in certain LDP (Livingston, 2003; Painter, 1984; Ryan, 2007). Unfortunately, there was no common and apparent rationale for this result. While not identical, the methodologies for the three evaluations were similar in that they used quasi-experimental research with 360 degree validated evaluation instruments. Control groups and pretest-posttest designs were used such that the methodologies for the evaluations seemed adequate and theoretically sound. Ryan (2007) suggested that the single day duration of the particular program may have been the problem but that characteristic was not found in the other programs.

The results of these three studies were not generalizable because of the nature of their respective methodologies and their respective intents to focus primarily on the evaluation of a particular program. Thus, one could not say with certainty that the findings of any of the three could be replicated or would be replicated elsewhere. Important to this research were the findings that development programs fail. It was as essential to understand the failures as it was to understand the successes. One plausible explanation for the results in the three aforementioned studies has been provided by Boyatzis (1993) who suggested that persons who participated in leadership development activities often did not become leaders because the effectiveness of the activities may have been a function of the appropriateness to their “mode of growth”, be it performance, learning or development. Boyatzis also indicated that

there was a difference between capability and commitment to be a leader, in that the latter involved a desire to use the competencies one possessed. Furthermore, Bandura (1997) linked inaction to self-efficacy and outcome expectations and suggested that action would only occur if self-efficacy beliefs and outcome expectations were indicative that the action was worthwhile (p. 24). Thus, an individual can possess the competencies but not

demonstrate leadership because they do not perceive the competencies to be relevant or sufficient. Programs need to encourage individuals to take on the role of leader, and affirm their belief in their ability to do so, not just improve their leadership skills. These

explanations and findings demonstrated the importance of determining appropriate program content and attributes.

Previous research suggested that the distinction between leader and leadership development and the appropriate program attributes determine the effectiveness of the leadership development intervention. Beneficial leadership development experiences included five components: self-efficacy, sociality, relevance or purposive nature of the task/activity and reflectivity that have been associated with providing the competencies that are critical for effective leadership (Olivares et al., 2007). While based on case study and not necessarily generalizable, research suggested that significant organizational outcomes arose as a result of a collective and emergent leadership development program intervention that focused on the collective, not just the individual leader (McGurk, 2010). Both of these findings are important to the context of the research and suggest factors that should be considered in the evaluation of the intended research. As in previous research, the

relationship between leadership self-efficacy and effective leadership development supports the importance of the research herein.

Avolio et al. (2009) were interested in whether the leadership theory underpinning the intervention made a difference. The leadership theories associated with 138 experimental and quasi-experimental studies were classified as traditional, new (to represent the

transformational schools of thought) and Pygmalion leadership. The relationship between the theory and the results of the intervention was determined. Participants subjected to

Pygmalion leadership interventions had a 79% chance of success compared to 64% for newer leadership theories (Avolio et al., 2009). However, both new and traditional theories had a moderately positive overall effect and overall their effects were not significantly different (Avolio et al., 2009). This was an important finding in that the choice of leadership theory did not appear to be a factor in the success of the program and thus improved the generalizability of results that considered a specific theory.

Kempster (2009) did not study a particular leadership development intervention but rather based his conclusions that formal leadership development programs had only a minimal impact on the overall leadership development of an individual on evidence gathered from a multifaceted qualitative study comprised largely of interviews of successful leaders.

Kempster acknowledged that formal programs provided value relative to developing the self- identity of the participant and in providing peer groups and networking opportunities. However, leadership salience, leadership self-efficacy and leadership identity were interrelated and essential components of leadership development which occurred through situational, enacted and observed learning opportunities over the course of one's lifetime or career (Kempster, 2009). Leadership development is characterized by a multitude of experiences and an ongoing process rather than something that can be achieved in a single intervention (Bilhuber Galli & Muller-Stewens, 2012; Van Velsor, Moxley, & Bunker,

2004). Thus, Kempster was not convinced that formal leadership development had value but rather suggested that informal opportunities, situational encounters, context dependent experiences and notable people influenced leadership development. Nonetheless, a well- structured formal development intervention can incorporate the attributes that Kempster determined as most important. Most notably, the participation of recognized leaders or relevant experts has characterized many interventions.

The aforementioned studies showed that participation in leadership development programs generally led to individual knowledge and behavioural outcomes. Moreover, leadership self-efficacy has been a prominent outcome. However, some programs worked, while others did not, yet there was no conclusive research that addressed why this was the case even though some evidence suggested that leadership development could not be attained in a single intervention. While the organization may have experienced performance improvements as a result of leadership development interventions, there was little evidence in the research presented for this outcome. However, when an organizational outcome was evident, the studies were deficient in providing any explanation as to why that had occurred. Furthermore, the research did not investigate what program elements led to the

organizational outcome. No study compared those programs with lower or no organizational effect to programs that yielded positive organizational outcomes to determine the

intervention factors that might have created the difference. More research is required to establish a metric for assessing leadership effectiveness, to develop a correlation between leadership behaviour and organizational outcomes, and to develop a causal relationship between leadership training and the development of leaders (Allio, 2006).