Legitimisation is the collective term that refers to the validity, reliability and generalisability of the collected data (Easterby-Smith, 1991). The purpose of the section was to establish the trustworthiness of the data and the rigour of the research strategy adopted for this study.
Central to the research design is evolution from Pre-Understanding to Understanding. Pre-understanding is the literature based theoretical (the theory of affordances) underpinnings of the research focus itself setting the scene from which your ontological stance and epistemological understanding shaped the conceptual framework of your research direction.
Understanding, however, evolves through how reflection over the period of research between the “definition of the topic and research question, existing literature and your data and its analysis” (Karlsson, 2009) offers greater insight from which more generalized knowledge claim and contribution can be made.
Karlsson (2009) identifies trustworthiness as the general criterion for quality research based upon four requirements: construct validity, internal validity, external validity and reliability.
Construct validity is concerned with ensuring that the measures used to research identified constructs will actually measure the concepts that they are intended to measure.
Internal Validity is concerned with causality of demonstrated relationships: that the relationship demonstrated is the result of what was measured and not as a result of any other factor(s).
External Validity is concerned with ensuring that the results are valid in similar settings outside the studied object: knowing whether findings can be generalized beyond the immediate case study.
Reliability is concerned with the extent to which a study can be repeated (Yin, 1994): that another researcher should reach the same conclusion in the same research setting.
Each criterion deserves explicit attention, discussed in relation to the research in the following sections.
4.3.8.1 Construct Valididty
The construction of a conceptual framework, developed as a way of visualising the theoretical literature that underpins the objective (management of the integrity of engineered product representations) of the research (Miles and Huberman, 1994), was central to demonstrating the validity and reliability of the research, from which contribution were to be made. The conceptual framework, as visualised in figure 1 of chapter 3, presented and defined five constructs. The validity of each was demonstrated by identifying the operational measures for exploring each. The five constructs and the related operational measures were as follows: (1) use environment, (2) artefact, (3) affordance subsystems, (4) action possibilities, and (5) total product affordances. Toward representing a methodological fit with the research focus, emphasis was placed upon understanding how the relationship between the constructs under investigation supported exploration of management of the external-internal affordance relationship phenomenon: The proposition behind the constructs of the conceptual framework was that the affordances of a product’s design is challenged by the different but related aspects informing managing of a product development to capture complementarity with a use environment.
Three tactics were applied to increase construct validity (Yin, 1994). First, data was collected from multiple sources and different methods were used to to obtain that data. This allowed for the triangulation of evidence on convergent meanings (Yin, 1994). Second, a prolonged engagement with each informant companies enabled a chain of
77
evidence to be established. Finally, according to Hirschman (1986), the people most capable of evaluating the completeness of the interpretation of that world view are those from whom data originated. Thus, on a regular basis, the gathered data, the interpretations were discussed with the research informants for review to ensure its adequacy and validity in portraying the phenomenon being explored. Upon completion of the findings chapter, the researcher presented each case company with findings pertaining to their company only. Each company was asked to read the findings and verify the validity of the data and the development process that they had participated in. If the company was unhappy with the findings, or if the researchers had missed or had misinterpreted something, the contact enabled these errors to become known to the researcher so that rectifications could be arranged. Company A verified approval through a phone discussion . Appendix F presents an email screen shot visualising the approval of the case write up from case company B’s perspective.
4.3.8.2 Internal Validity
Internal validity only applies to explanatory or causal studies and not descriptive or exploratory studies (Yin, 1994). This research was exploratory in nature. Internal validity was not applicable to this research and therefore was not discussed.
4.3.8.3 External Validity
Critics generally state that single cases offer a poor basis for generalising (Yin, 1994), and that theory must be tested through replications of findings. This research was based upon the multiple case strategy of literal replication, focused on product affordance improvements and the design challenge of managing complementarity of internal and external subsystem design considerations relative to specific product use environments. However, one cannot assume observations are applicable to all products. Generalisation is not automatic (yin, 1994). The researchers has provided, through observations recorded, rich description that would enable other researchers to assess conclusions drawn and determine whether the study could become “the vehicle for examining other cases” (Yin, 1994).
4.3.8.4 Reliability
According to Yin (1994), a good guideline for doing case studies, is conducting research so that an auditor could repeat the procedures and arrive at the same result. For this study, two tactics were applied to establish reliability.
First, the reliability of findings were addressed by documenting the multiple sources of evidence & triangulation procedures used for data collection and providing a documented audit trail that demonstrated how the exact same same method as applied in case one were also applied in case two.
Second, the theoretical and philosophical basis for the research, along with the process of inquiry, the collected raw data, the findings, interpretations and recommendations were all presented to an auditor (Lincoln & Guba, 1985) or external observer (Yin, 1995) for examination. Yin (1995) argued that the process should be tight enough so that the conclusions obtained be “assuredly the same evidence that was collected at the scene…during the data collection process; conversely, no original evidence should have been lost, through carelessness or bias”. The task of the auditor was to review the quality of the research design and findings to assess the triangulation of evidence and to confirm or disconfirm that the conclusions drawnderived at flowed from the gathered data (Hirschman, 1986; Lincoln & Guba, 1985). To facilitate such an examination, an audit trail from the philosophical, theoretical and methodological backgrounds, to raw data, interview notes and transcripts, to analytical procedures, to interpretations and conclusions were all presented for review.