Actividades desarrolladas por los participantes

The most accurate approach to isolate the impact of leadership development is the use of control groups in an experimental design process (Wang, 2002). This approach involves the use of an experi-mental group that participates in leadership development and a control group that does not. The composition of both groups should be as similar as possible and, if feasible, the selection of participants for each group should be on a random basis. When this is possible and both groups are subjected to the same environmental influences, the difference in the performance of the two groups can be attrib-uted to the leadership development program.

As illustrated in Figure 5-2, the control group and experimental group do not necessarily have preprogram measurements. Measure-ments are taken after the program is implemented. The difference in the performance of the two groups shows the amount of improve-ment that is directly related to the leadership developimprove-ment program.

Control group arrangements appear in many settings, including both private and public sectors. For example, a turnover reduction program for communication specialists in a government agency used both a control group and an experimental group (Phillips and Phillips, 2002). The experimental group was compiled of individu-als in a special program designed to allow participants to achieve a master’s degree in information science on agency time and at agency expense. The control group was carefully selected to match up with the experimental group in terms of job title, tenure with the agency, and the college degree obtained. The control/experimental group dif-ferences were very dramatic, showing the impact of the retention solution program.

One caution—the use of control groups may create an image that the leadership development staff is creating a laboratory setting, which can cause a problem for some administrators and executives.

To avoid this stigma, some organizations run a program using pilot

Isolating the Effects of a Leadership Development Program 121

Control Group

(Untrained) Measurement

Experimental Group Trained

Program Measurement

Figure 5-2. Post-test only, control group design.

participants as the experimental group and do not inform the non-participating control group.

Advantages and Disadvantages

The control group process does have some inherent problems that may make it difficult to apply in practice. The first major problem is that the process is inappropriate for many situations. For some types of leadership development programs, it is not proper to with-hold development from one particular group while leadership devel-opment is given to another. This is particularly important for critical skills that are needed immediately on the job.

This particular barrier keeps many control groups from being implemented. Management is not willing to withhold leadership development in one area to see how it works in another. However, in practice, there are many opportunities for a natural control group agreement to develop in situations where leadership development is implemented throughout an organization. If it will take several months for everyone in the organization to receive the leadership development, there may be enough time for a parallel comparison between the initial group participating in the program and the last group participating in the program. In these cases, it is critical to ensure that the groups are matched as closely as possible so that the first two groups are very similar to the last two groups. These naturally occurring control groups often exist in major leadership development program implementation. The challenge is to address this issue early enough to influence the implementation schedule so that similar groups can be used in the comparison.

The second major problem is the selection of the groups. From a practical perspective it is virtually impossible to have identical control and experimental groups. Dozens of factors can affect employee performance, some of them individual and others contex-tual. To tackle the issue on a practical basis, it is best to select three to five variables that will have the greatest influence on performance.

A third problem with the control group arrangement is contam-ination, which can develop when participants in the leadership development program instruct others in the control group. Some-times the reverse situation occurs when members of the control group model the behavior from the trained group. In either case, the experiment becomes contaminated because the influence filters to the control group. This can be minimized by ensuring that control groups and experimental groups are at different locations, have

different shifts, or are on different floors in the same building. When this is not possible, it is sometimes helpful to explain to both groups that one group will participate in a leadership development program now and another will participate at a later date. Also, it may be helpful to appeal to the sense of responsibility of those participat-ing in the program and ask them not to share the information with others.

Closely related to the previous problem is the issue of time. The longer a control group and experimental group comparison oper-ates, the likelihood of other influences affecting the results increases.

More variables will enter into the situation, contaminating the results. On the other end of the scale, there must be enough time so that a clear pattern can emerge between the two groups. Thus, the timing for control group comparisons must strike a delicate balance of waiting long enough for their performance differences to show, but not too long so that the results become seriously contaminated.

A fifth problem occurs when the different groups function under different environmental influences. Because they may be in different locations, the groups may have different environmental influences.

Sometimes the selection of the groups can help prevent this problem from occurring. Also, using more groups than necessary and dis-carding those with some environmental differences is another tactic.

A sixth problem with using control groups is that it may appear to be too research oriented for most business organizations. For example, management may not want to take the time to experiment before proceeding with a program or they may not want to with-hold leadership development from a group just to measure the impact of an experimental program. Because of this concern, some leadership development practitioners do not entertain the idea of using control groups. When the process is used, however, some organizations conduct it with pilot participants as the experimental group and nonparticipants as the control group. Under this arrange-ment, the control group is not informed of their control group status.

Because this is an effective approach for isolating the impact of a leadership development program, it should be considered as a strat-egy when a major ROI impact study is planned. In these situations it is important for the program impact to be isolated to a high level of accuracy; the primary advantage of the control group process is accuracy. About one-third of the more than 100 published studies on the ROI methodology use the control group process.

Isolating the Effects of a Leadership Development Program 123