user issues concerning the IS and IT in general. User training is also provided at intervals through out the year. A combination of in-house and external con- sultants support and maintain the IS. The ORACLE DBMS (Enterprise Edition 10g) is hosted on a UNIX server, an ORACLE forms server is employed for the application. User authentication is via the network operating system and sepa- rate individual user login for Airline IS application. The role groups, correspond to the stakeholders outlined in section 6.2. Access to the IS is via database soft- ware on the DBMS server (MIS stakeholders) web client on the desktop PC and mobile devices (PDAs and Smartphones).
6.3
Evaluation Approach - Method Application
After the build phase of our proposed method it is necessary to evaluate the constructed artefact. Data from the experiments conducted with the Library IS assisted in this process, as it identified suggested variances in IQ depending on IS situation. This information was used in conjunction with our method and in particular with respect to the road map for method implementation. Pfeiffer and Niehaves [114], evaluation of conceptual models adopt a holistic approach to the evaluation of the artefact, consisting of an analysis of artefacts for evaluation and criteria for their evaluation. The holistic approach adopted by the conceptual model is suited to our research as it facilitates a comprehensive evaluation of our method, which is similar in philosophy to the TDQM approach that we adopted throughout our research.
The first component of the model addresses the IT artefacts, including con- structs, methods, models and instantiations. The conceptual model requires that methods explain in detail the process of problem solving, in conjunction with guidelines to provide for a solution. The second component of the model de- tails the structure, evaluation criteria and evaluation approach that collectively provide for a holistic approach to evaluating the IT artefact [114].
• Structure of the Artefact: Determines the configurational characteristics necessary to enable the evaluation of the IT artefact. Based on this struc-
ture, all required information about the artefact can be deduced. The structure represents the information space the artefact spans.
• Evaluation Criteria: Specifies the dimensions of the information space which are relevant for determining utility of the artefact. These criteria may differ on the purpose of the evaluation.
• Evaluation Approach: Defines all roles concerned with the assessment and the manner of handling the evaluation. The result is a decision on whether or not the artefact meets the evaluation criteria based on the available information.
The conceptual model also outlines examples of corresponding research results for each of the constructs. Table 6.1 describes those that are appropriate to the method artefact. We employed these to inform our approach to evaluation of our method for diverse IS situations and IQ.
Table 6.1: Conceptual Model to Evaluate IT Artefacts (Methods) Artefacts Structure Evaluation -
Criteria
Evaluation - Approaches Method Process Based
Meta Model, Intended Applica- tions, Conditions of Applicability, Products and Results of the Application Appropriateness, Completeness, Consistency Lab Re- search, Field Enquiries, Surveys, Case Studies, Ac- tion Research, Practice, De- scriptions, Interpretative Research
Pfeiffer et al. [114] state that a core constituent of a method is given by a process model, which must describe how to reach the objective of the method.
6.3 Evaluation Approach - Method Application
Becker et al. [17] ascertain that the model has to explicitly state the product and results of its application as well as constructs used in the context. In the case of our method this is systematically mapped using Business Process Modelling.
Critically Greiffenberg [61] points out that in order to appraise the applicabil- ity of the method it must describe its conditions and intended scope of application. Greiffenberg [61] also outlined the criteria to evaluate methods, and three criteria are recommended to evaluate the artefact:
• Appropriateness: Verifies whether the method is efficient, well structured and easy to apply.
• Complete: Describes the methods inputs and outputs as well as its processes and relations.
• Consistency: Satisfied if all the method elements are mutually compatible. The evaluation of methods can be completed by employing field surveys, case studies and action research.
Gericke et al. [59] believe that the validation should be realised within different steps. Initially, the utility of the identified method fragments should be proven. Next the identified roles of possible method users and the developed method con- figuration should be evaluated regarding their appropriateness. Finally Gericke et al. [59] suggests that the interplay of the different method fragments, i.e. the whole method should be evaluated by applying it in real implementation projects. Design Science research [68, 110] requires that upon construction of the arte- fact evaluation must be clearly demonstrated. The validation of our method is performed through a verification of its implementation. We examined the impact that diverse IS situations had on the perception of IQ. We initially examined the field of IQ and associated frameworks and dimensions. In order to address and overcome the problem we constructed a method to aid in the implementation of an IQ framework. Experiments further enhanced this, by demonstrating that perceptions of IQ are affected by the diverse situation of access.
The Library IS experiment provided us with an excellent opportunity to test our hypotheses with respect to the individual IQ dimensions. It also acted as our development ground for the method fragments, business processes and models.
This involved prototyping, workshops, interviews and focus groups, and was iter- ative in nature with many of the participants being central to the process. This allowed us to apply and revisit method fragments regularly until we refined the individual fragments. This involved a major rework of many elements along with much discussion and suggestion from stakeholders and IS professionals. We de- cided to conduct the assessment of our method with the airline IS because there was a greater ability to assess the method independent of a specific domain that it was developed in.
An action research approach is adopted for this portion of our research, as we are active participants in the intervention or action studied. There is a re- quirement to heavily interact with the stakeholders of airline IS; therefore an organisational rather than a machine solution is deemed appropriate. This lent itself towards a methodology where the affect on the stakeholders is a central tenet of the philosophy. Checkland [28] describes it as holistic rather than re- ductionist. Baskerville and Harper [14] describe the ideal domain of the action research method is characterised by a social setting where:
• The researcher is actively involved, with expected benefit for both researcher and organisation.
• The knowledge obtained can be immediately applied, there is not the sense of the detached observer, but that of an active participant wishing to utilize any new knowledge based on an explicit, clear conceptual framework. • The research is a (typically cyclical) process linking theory and practice. Susman and Evered [138], as described in Chapter Three detail a five phase, cyclical research process. The approach first requires the establishment of a client- system infrastructure or research environment. Then, five identifiable phases are iterated:
• Diagnosing • Action Planning • Action Taking