Internet, progreso y libertad de expresión.

The research model was tested using a laboratory experiment in a 2x2 fully factored design in which Role Prescription and Credibility were independently manipulated .

3.3.1 Development of the Materials and Instruments

The scenarios utilized (Appendices A-D) were modified versions of the scenarios used in paper 1. For this study, language was introduced to manipulate the Role Prescription of the bad news reporter. In the positive manipulation, the bad news reporter was an auditor as in paper 1. S/he was also described as having the specific responsibility to report on project problems. In the negative manipulation, the bad news reporter is a peer project leader who volunteers to review the project. S/he is specifically described as not role prescribed to review projects and report on their findings. Additionally, the scenario was modified to specifically state that insufficient time was left in the schedule to do further testing and still make the implementation date. This change was made because in the study reported in paper 1, some subjects chose to delay and then commented that they would do the testing prior to the implementation

The instrument was improved by rewording one question to match the revision of the scenario and adding two more items measuring Credibility. I did this to improve the reliability of the construct over that reported in paper 1, which was somewhat low, although acceptable. I drew on the source credibility literature (Pornpitakpan 2004) to add two measurement items dealing with the key dimension of Credibility:

trustworthiness and expertise. Additionally, I revised an item to reflect the effect of organizational power on Credibility by asking about the bad news reporter’s perception in the organization. I added items on Risk Perception, Risk Propensity and Role

Prescription to the instrument. The risk items were adapted from Sitkin and

Weingart(1995) to fit into a seven point Likert scale format. I added a single item to assess the subject’s perception of the Role Prescription of the bad news reporter.

3.3.2 Scenario

These scenarios (Appendices A-D) were modeled in part on the Providian Trust case (McFarlan, 1997) and the Challenger Disaster scenario (Hauptman and Iwaki 1990). In both cases I saw some common features that are incorporated into this scenario. In both cases, there were pressures external to the decision maker that moved the decision maker to continue the current course of action. There was also a decision maker who could not perform the evaluation on his/her own and a bad news reporter or reporters who could not communicate the potential disaster situation in terms that unambiguously indicated a coming disaster. Additionally, the decision maker was led to believe there were severe time constraints on the length of time to evaluate the situation and make a

decision. Each of these four items is included in the scenarios.

Contents of the Scenario. The subjects were instructed to play the role of a project manager responsible for development of a new application system to be put into production. The development team has indicated that the project is complete. A bad news reporter has reviewed the project and given a negative report on its readiness for production. To manipulate Role Prescription, the bad news reporter is described as an internal auditor, whose job is to assess projects and report bad news in the positive manipulation or a project manager from a different development team whose role specifically does not include examination of projects and reporting bad news. The bad news reporter does not provide understandable reasons for why S/he believes the project would fail and the subjects are thus placed in a situation in which they must decide to what extent they should rely on the assertions of others. The manipulation of Credibility is performed by describing the bad news reporter as expert and highly regarded in the organization (positive manipulation) or inexpert and not highly regarded (negative manipulation). Additionally, the subject is informed that the environment in which they work stresses the meeting of dates and budgets by word and reward and punishment system. Additionally, the management chain above the subject is said to be pressing for on time implementation of the system. The subject could choose to delay the project and face the known problem of dealing with their management’s

expectations, or the subject could choose to implement the system and face the uncertain prospect of a catastrophic problem. The subject is then asked to decide whether s/he would continue with the scheduled deployment or delay the project for further testing.

3.3.3 Procedure

The experiment was administered in a classroom setting. Access was obtained by permission of the instructors. The time obtained was usually in the last part of a class or in time remaining after the administration of a test. Prior to administration of the

instruments, instructions for completing the instrument were given to the class. Subjects were informed that this was an experiment on decision making and that their

participation was completely voluntary and they could terminate their involvement at any time. They were randomly selected for one of the treatment groups. Subjects were instructed to read a scenario describing an information systems project and then to make a decision as to whether to move the project into production (i.e., deployment) or delay the deployment to allow time for further testing and then answer several questions about their decision. Following their completion of the survey, they returned the materials to the administrator and left the classroom. Subjects had approximately 20-30 minutes to complete the experiment. The subjects were not time constrained in responding to the instrument, the time constraint arose from the text, which indicated that there was no time to test prior to the implementation date.

3.3.4 Subjects

This experiment was conducted using undergraduate student subjects. The scenario was constructed so as to place the subject in the role of a recent graduate, which provides a decision making context that is close to what might be experienced by the subject shortly after entering the work force. While the appropriateness of student subjects has been debated, there is ample precedent for using student subjects in studies with organizational decision making tasks (Sitkin and Weingart 1995) and, specifically, decisions associated with project management (Harrison and Harrell 1993; Smith, et al. 2001). From a philosophic perspective, the question comes down to the

issue of how much emphasis to place on external validity: should the subjects of the experiment function as exact surrogates for practitioners? While some scholars insist on having external validity for every study (Lynch 1982; 1983), Calder, Phillips and Tybout (1981; 1982; 1983) argue that external validity is not a requirement for a rigorous theory test. Based on what the study is designed to demonstrate, the requirement for external validity varies. If the study is to show how the theory can be applied to real world situations, then there is a requirement for the subjects, testing and variables to be analogous to the real world. If, however, the study is designed to be a test of theory, then the requirement is to produce the strongest possible test in an attempt to falsify the theory. External validity is not required and may be sacrificed to achieve internal and construct validity. In this case homogenous samples such as student subjects and laboratory experiments are stronger. For theory testing, external validity is best

addressed during theory development. After internal validity is achieved, external validity is addressed by testing across multiple contexts. Cook and Campbell (1979) also make this point saying:

The priority among validity types varies with the kind of research being conducted. For persons interested in theory testing, it is almost as important to show that the variables involved in the research are construct A and B (construct validity) as it is to show that the relationship is causal and goes from one variable to the other (internal validity). Few theories specify crucial target settings, populations, or times to or across which generalization is desired. Consequently, external validity is of relatively little importance. In practice, it is often sacrificed for the greater statistical power that comes through having isolated settings, standardized procedures and homogenous respondent populations. For investigators with theoretical interests our estimate is that the types of validity, in order of importance, are probably internal, construct, statistical conclusion, and external validity (p. 83)

Therefore in this situation, the question is whether student surrogates act similarly enough to actual practitioners to provide internal validity. Ashton and Kramer (1980) citing Zelditch and Evan (1962), indicate that students are to the behavioral researcher what the fruit fly is to the geneticist, a being that is different from the one desired to be studied but has a mechanism that operates in fundamentally the same manner as the target being. The question is therefore, whether the decision making processes of students are sufficiently similar to that of experienced professionals to enable a similar set of responses to be received in a decision situation. This question is an empirical one (Liyanarachchi and Milne 2005).

Looking at the empirical literature, Birnberg and Nath (1968) indicate that student subjects are likely to differ from real world subjects in terms of skills, experience and personality traits. The effects of experience are manifested in terms of enhanced facility with and development of job related skills, routinized patterns of behavior and attitudes toward certain situations. Some empirical studies have shown support for using students as surrogates for managers in studies that focus on decision making and which do not require deep knowledge of particular domain. Ashton and Kramer (1980) in a review of the literature to that date found at least moderate support for using student surrogates in decision making tasks although not in studies of attitudes. An example of the disparity in attitudes is shown by a study of attitudes toward corporate social responsibility by

Ibrahim, Howard, and Angelides (2008). In this study, students were found to be more attuned to the ethical and philanthropic areas of social responsibility, while practicing managers were more attuned to the economic responsibilities. This shows the

differences in attitudes between students and managers as an effect of experience. Having been subject to the pressures of the marketplace, managers are necessarily more attuned to economic reality while students who haven’t had that experience put more stress on ethics and philanthropy (Ibrahim, et al. 2008).

Ashton and Kramer report that psychological studies show that students and real world decision makers show “extremely similar information processing characteristics and biases” (p.3). In an experiment to test students as surrogates for experienced auditors in decision tasks, they found that students matched the auditors’ decisions 67% of the time and only 7-13% of the difference was attributable to experience. Further, they found that students and auditors responded similarly in a number of different situations. They argue that students can act as surrogates in terms of theoretical studies as opposed to

application studies, a conclusion similar to that described by Calder, et al. In more recent work, Remus (1986), reported no differences in decision making between

students and managers in the context of production scheduling. Locke (1986,p. 6) notes that “both college students and employees appear to respond similarly to goals,

feedback, incentives, participation, and so forth, perhaps because the similarities among these subjects (such as in values) are more crucial than their differences.” Liyanarachchi and Milne (2005) found that accounting students making experimental investment

decisions react similarly to practitioners with respect to environmental disclosures. In this discussion, an important recent study that must be considered is that of Chang and Ho (2004). Chang and Ho ran an experiment comparing student and manager escalation behavior when project completion and market information was manipulated. Their results show that managers and students had statistically the same likelihood to continue projects on 60% of the tests with managers having a higher likelihood on the remainder. More importantly, the direction of the responses was similar between the managers and students. Both tended to have lower likelihood of completion when the project was less complete than when it was more complete. The managers and students had different responses and response patterns for the funds allocation issues. These patterns are highly influenced by attitudes developed by managers’ experience in the business world in which they learn that meeting profitability targets is a key to success. Students, however, without that experience, have not been so conditioned and therefore have different attitudes towards allocation of funds. The results of the Chang and Ho study are therefore consistent with the Ibrahim study. Thus we see that in an escalation situation, students responded similarly to practitioners in the escalation decision where experience was not a factor. In the funds allocation issues, where experience is a factor, their choices were different.

Based on the literature, it would appear that student subjects can stand as surrogates for managers in decision making studies where economic issues are not involved. The studies surveyed show that students are in statistical agreement with the responses provided by managers approximately 60% of the time and the rest of the time while statistically different follow the general trend of the response pattern.

3.3.5

Operationalization of Variables

With two exceptions, the constructs in the research model were operationalized using multi-item measures with Likert-type or semantic differential scales. Decision and Role Prescription were measured with single item measures. Relevance was measured with four items taken from paper 1. Credibility was measured with three items from paper 1 with two new items designed by the authors. Risk perception and Risk Propensity were measured using items adapted from Sitkin and Weingart (1995). Details of the

operationalization of the items are found in Appendix F.