Reliability is the extent to which a procedure will produce the same results under constant conditions (Bell, 1992, Neuman, 2003). In the case of this study, the reliability of the research results entailed whether or not the same findings would occur if the study were repeated in the same manner. There were however, difficulties in assessing and testing this stipulation because of the qualitative nature of the information collected and rapid change in the area under investigation.
3.12.1 Reliability
Great care was taken at the planning, implementing and analysis stages to ensure reliability was taken into consideration. (Benbasat et al., 1987) states that a clear description of the data sources and the manner in which they contribute to the overall findings of a study is an important aspect to the reliability and validity of the results. For this reason, a clear description of the data sources and methods used to gather those sources have been provided. Data collected using interviews were open to problems such as interview bias, misdirected prompting and issues of question wording. These issues were noted during the interview process and attempts were made to minimise these effects, although it is unlikely that interference was eradicated completely.
With regard to the results from observation techniques, the issues of reliability are somewhat easier to assess than data collected via interviews. Because much of what was observed was inanimate and static (such as technological deployments, computer and information systems, intranet solutions) the issues that typically affect the reliability of observation results such as potential recorder bias and obtrusive influence did not apply. Subsequently, these observations have high reliability.
3.12.2 Validity
Validity describes whether an item measures or describes what it is supposed to measure or describe (Bell, 1992). It is a much more complex concept than reliability
and there are many variations and sub-divisions to which researchers can investigate in attempts at ensuring validity of their results. (Bell, 1992) states that researchers involved with smaller projects without complex testing or measurements need not investigate the concept of validity too thoroughly but should examine results and methods critically. Noting this, a brief dialogue of the aspects of validity is discussed.
Face Validity
The easiest aspect to achieve, and the most basic kind of validity is face validity. Face validity is a judgement by the scientific community as to whether or not the indicator really measures the construct (Neuman, 2003). This aspect relies on the fact that readers will accept the definition and measurement fit of the instrument presented.
Content Validity
Content validity addresses whether or not a definition is represented within a measure. A conceptional definition contains a ‘space’ for thoughts and ideas that the researcher put forward that surround and pertain to the construct. An example in this research would be the measure of perception of the level of satisfaction that students may have concerning E-learning that is held by the teacher. How valid is the definition of student satisfaction? Are the views expressed indicative of the thoughts of the students? Does the definition of student satisfaction need to be expanded or narrowed in an attempt to fulfil the requirements of the research and thus be eligible for inclusion in the study?
Criterion Validity
This form of validity uses a set standard or criterion, cross referenced to the construct, to indicate the level of validity that may be compared to a similar construct that has been known to be acceptable. A concurrent validity indicates that the construct agrees with pre-existing values confirming its validity, where predictive validity conforms to logically construed future values or events relative to the construct.
Construct Validity
Validity means truthful. It refers to the bridge between construct and the data. Qualitative researchers are more interested in authenticity than validity (Neuman, 2003). However, (Peraklya, 1997) argues that construct validity is central to the overall validity of research. Construct validity is concerned with the relationship between a theoretical model and the observations made by the researcher. This is particularly relevant in this research, where the discussion of theoretical models and themes identified from interviews form the main part of the results. If the discussion of these theoretical concepts bears little relevance to the factual realities observed in the field, the findings of the research will be invalid and void.
To increase validity and to ensure accuracy, follow-up e-mails were used to discuss and clarify topics of discussion. Where relevant, portions of the research that discussed systems and observations were sent using e-mail to the respective interviewees for their confirmation that analysis and descriptions of observed models were correct. This ensured that what was stated in the research was factual and accurate.
3.12.3 Validity and the Generalisation of Findings
Another facet of validity relates to the generalisation of research findings. This topic has already been discussed with regards to sampling methods. The result of this research was produced from a relatively small sample population.
Within the research, the data has been kept as pure and free of bias as possible. Definitions of measures used in the interpretive analysis stages have been done from a neutral stance as possible to ensure no bias from the researchers viewpoints or previous life experiences. Any interpreted qualification of data is therefore based on observed and implied information from participants involved with the study and should be recognisable as being both conceivable and verifiable by readers of the research.
However, as previously stated, the intention of this research was not to produce definitive results that could be generalised and applied elsewhere. Therefore it is suggested that the findings presented are valid within the context discussed.