Our experiment examined users perception of IQ of diverse situations for a single IS within an organisation, however despite this limitation we contend that our findings have relevance to the wider context of E-Commerce. The perception of IQ can be viewed from the perspective of the users trust or belief in the IS. Our experiment findings demonstrate that the perceptions of IQ vary depending on the IS situation. This is greater for some dimensions than others (table 4.2).
The trust that users place in a web-site is essential for success. Maximis- ing trust and minimizing risk are the critical factors that dictate the long-term success of a web application. In essence trust encourages relationships and con- sequently economic activity through cooperative transactions that may be at an individual or corporation level [81]. It has been identified as a key factor in many studies and has been examined from a range of perspectives including technical, social, psychological, economic and behavioural [105]. Trust as a phenomenon can be difficult to observe and similar to IQ it is multi-dimensional in nature. Frameworks and models have evolved [113] in attempt to classify, measure and improve trust and although accuracy, believability and other dimensions of trust are examined in the literature their relationship with diverse IS situations remains an open research question.
The results of our experiment demonstrate that the IS situation is a factor that must be considered when measuring user perception and trust and though objective measures of IQ may return values for intrinsic dimensions such as ac- curacy or completeness, the same outcome may not be perceived by the user of a particular IS situation. Objective measures we argue should therefore be used in conjunction with subjective measures for each diverse IS situations.
*WS = Workstation *MOB = Mobile *WEB PC = Browser launched appli- cation on Personal Computer
Table 4.2: Significance IS Situation Dimension WS V MOB WEB PC V
MOB WS V WEB PC Believability r=-0.64(L) r=-0.55(L) r=-0.29(S) Accuracy r=-0.28(S) r=-0.21(S) r=-0.19(S) Objectivity r=-0.61(L) r=-0.49(M) r=-0.09(S) Value-Added r=-0.31(M) r=-0.28(S) r=-0.23(S) Relevancy r=-0.27(S) r=-0.22(S) r=-0.14(S) Timeliness r=-0.67(L) r=-0.58(L) r=-0.24(S) Completeness r=-0.59(L) r=-0.48(M) r=-0.08(S) Appropriate Amount of Data r=-0.31(M) r=-0.28(S) r=-0.29(S) Interpretability r=-0.67(L) r=-0.52(L) r=-0.28(S) Ease-of-Understanding r=-0.71(L) r=-0.66(L) r=-0.21(S) Representational Consistency r=-0.64(L) r=-0.48(M) r=-0.12(S) Concise Representation r=-0.52(L) r=-0.54(L) r=-0.17(S) Accessibility r=-0.68(L) r=-0.61(L) r=-0.31(M) Access Security r=-0.60(L) r=-0.48(M) r=-0.28(S)
4.4 Experiment Analysis Figure 4.7: Significance of IS Situation
4.5
Diverse IS Situations and IQ Dimensions
The variance between IS situations requires that those charged with ensuring IQ for the IS take it into consideration when calculating the overall level of IQ. There are a number of ways that this variance can be achieved depending on the IS. We demonstrated that it is possible to measure empirically the perception of IQ and calculate the variance Cohens [44] value and situation analysis.
In order to aggregate single IQ measures, researchers have proposed a num- ber of options; often underlying a weighted aggregate of single values for IQ dimensions [147]. Although, some researchers have attempted to identify IQ value curves and trade-offs by analyzing the potential impacts of IQ [137], much research still measure the overall impact of IQ as a weighted aggregate. For exam- ple, a principle measure of the weighed sum of all the criteria (IQCi) is illustrated
in equation 4.4 as IQ = n X i−1 αiIQCi where ∀αi : 0 ≤ αi ≤ 1 n X i−1 αi = 1 (4.4)
The weight, or priority of each IQ criteria (IQCi) is represented by α i. How-
ever, most frameworks provide limited assistance to define the weights for IQ dimensions. Furthermore, most frameworks do not provide any guidelines on how to apply the framework and the IQ aggregation to diverse IS situations. In order to provide indications for aggregating IQ measures, weightings need to be assigned to reflect the perception of the same information from diverse situations or at the very least the bias should be recorded. We address this, proposing that the IS situation factor be represented when aggregating IQ measurement by for example the weighted sum measure [54]. Indeed each weight used for aggregating IQ measures needs to be IS situation dependent. This is reflected in revising the traditional weighted sum aggregation, by including IS situational factors c as illustrated in equation 4.5. IQc= n X i−1 αciIQCi where ∀αi : 0 ≤ αi ≤ 1 n X i−1 αi = 1 (4.5)
4.5 Diverse IS Situations and IQ Dimensions
The weight or priority of each IQ criteria (IQCi) is represented by αci which
considers the IS situational adjustment for this IQ criterion. This allows for aggregation and comparison of various results of IQ assessments within different IS environments.
In traditional IQ approaches and frameworks, priorities and weights for IQ dimensions, where assigned are completed without considering fully the impact of diverse IS situations. However, as our experiment demonstrates that there is a relationship between the IS situation and the corresponding satisfaction with IQ dimensions. Although an organisation may consider individual dimensions to be particularly important and consequently assign an individual weighting to their significance, generally completed by means of survey instrument among managers and users this alone we argue is not sufficient. The IS situation and in particular the manner by which it is accessed it has been demonstrated to be a significant factor. The ranking or weighting of IQ needs to take this into consideration. The traditional method of applying a weighted sum measure irrespective of IS situation requires refinement in order to achieve a normalized IQ score.
Figure 4.7 illustrates the need to consider the concept of normalizing IQ for a situational factors on IQ assessment. Our experiment demonstrates that the sat- isfaction with the IQ dimensions is dependent on the IS situation from which the IS is accessed. The accessing of the same information from diverse IS situations produces different results with respect to the level of IQ. This presents us with the challenge of obtaining an accurate reflection IQ for IS with diverse access, which we address in Chapter Five by the construction of our method.