Curso de Gestión de la Calidad

The results of the survey and case studies will be discussed in the following chapters. Some data analysis principles, though, guided the data analysis and have bearing on the results. This section explains the quantitative and qualitative data analysis principles, including reliability and validity aspects.

3.4.1 Quantitative

Surveys can be defined as a way to study populations by selecting samples from a population in order to quantify the “incidence, distribution, and interrelations of sociological and psychological variables”

(Kerlinger & Lee, 2000, p. 599). Sociological variables include social groupings, such as age, income, education and race. Psychological variables include opinions, attitudes and behaviour. The relationship between and within each variable type is studied by surveys.

The dominant answer option in the survey was on a Likert scale. Kerlinger and Lee (2000) said that “a summated rating scale (one type of which is called a Likert-type scale) is a set of attitude items, all of which are considered of approximately equal “attitude value,” and to each of which the participants respond with degrees of agreement or disagreement (intensity)” (p. 712). There are, however, an increased risk of response set variance, or that individuals tend to respond differently to extremes/neutral options which can affect validity of responses. This tendency will be discussed under the limitations of the study in Chapter 6.

All Likert scale items were judged to be interval scales. Graziano and Raulin (2000) defines an interval scale as one which includes identity, an order of magnitude and equal intervals. With interval scales parametric testing can be performed. Parametric testing, however, is usually dependant on larger sample sizes. For this study, with a sample size below 100, non-parametric tests were preferred. The data analyses were performed on the Statistica data analysis package and in consultation with the Centre for Statistical Consultations at Stellenbosch University.

3.4.1.1 Reliability and validity

Reliability is defined as an “index of the consistency of a measuring instrument in repeatedly providing the same score for a given participant” (Graziano & Raulin, 2000, p. 431). Surveys are limited by their structured nature. It is difficult to control for subject effects, whereby, for example, “participants do their best to be good subjects” (Graziano & Raulin, 2000, p. 195), which may lead to the confounding of results. For this reason, controls needed to be built into the survey, such as standardised instructions, limited contact with the researcher, anonymity of the participants and the assurance of confidentiality. The researcher also obtained information on personal and individual characteristics (such as gender and level of education) in order to explain the population as thoroughly as possible.

Validity “refers to the methodological and/or conceptual soundness of research” (Graziano & Raulin, 2000, p. 436). Sapsford (1999, paraphrased from p. 139) lists seven forms of validity in measurement:

• face validity: looks valid;

• concurrent: correlates with established measures of the same thing;

• predictive: test correctly related to an outcome;

• criterion-related: correctly identifies between groups;

• uni-dimensionality: when testing one concept there should be multiple correlations between items and total score;

• reliability: scores should measure a relatively stable concept;

• construct: convergent and discriminant validity. The power of the test to give different scores for theoretically different people (discriminant) and same scores for theoretically similar people (convergent).

For survey research, validity means the survey should actually measure what it reports to measure.

Face validity was established for the survey through a thorough pre-testing of the survey with academic and industry experts. Concurrent validity was partly established due to the use of four academic source documents that all measured the same thing to formulate the survey questions. Predictive validity cannot be ensured with a survey, although some questions did test attitude against actions taken by the company. The survey sample was rather similar, but some between group differences on the independent variables leads one to assume some criterion-related validity. Uni-dimensional validity was tested on related variables and found to correlate, whereas construct validity is assume through the thoroughness in the research design in identifying valid and distinct constructs.

Threats to validity, however, still remain for survey research. A major threat to validity in attitude testing is that expressed attitudes and observed behaviour may not correlate (Wicker, as cited in Sapsford, 1999). Kerlinger and Lee (2000) warn that responses to mailed questionnaires (or similarily web-based questionnaires) are poor and responses cannot be verified. This yields a serious threat to generalisibility of the results of the sample to the population of this type of design. Some survey design issues may also affect validity of results. These threats will be considered in Chapter 6 when limitations to the study are discussed.

Survey research has inherent limits. This method was supplemented with the qualitative enquiry.

Kerlinger and Lee (2000, p. 601) declare that “the best survey research uses the personal interview as the principal method of gathering information”.

3.4.2 Qualitative

The case studies include both primary text data and secondary text data. Primary text data comes in the form of the interview transcriptions or textual data (Eriksson & Kovalainen, 2008):

The usefulness and relevance of textual data in qualitative business research is traditionally based on the idea of transparency. This means that texts are considered to represent directly what is being studied. In other words, texts are treated as suitable objects of analysis because we believe in their ability to tell us about the people and issues that they represent. (p. 89)

Secondary text data comes in the form of company policy documents that relate to disability management. These policy documents were obtained via the main contact person in the company, in this instance, the head of HR or via the company website.

The use of different sources of data (such as interviews and documents) is highly recommended for case studies (Yin, 2009). Data triangulation describes the process of using multiple data sources to describe and support the findings of the case study (Yin, 2009). The guiding data analysis principles of the qualitative data analysis will now be discussed.

3.4.2.1 Data analysis

The first step in data analysis of interviews is transcribing the interviews “from oral to a written mode”

(Kvale & Brinkmann, 2009, p. 180). Transcription types and content are based on the choice of research design and practical issues. A research design that requires discourse analysis, for example, requires verbatim transcriptions (Kvale & Brinkmann, 2009). As the current study is more concerned with facts and conceptual clarification, verbatim transcriptions were foregone for written style. This means that ease of reading the transcribe data were superior to recording responses verbatim (i.e. word for word, including false starts, repetition etc.) (Kvale & Brinkmann, 2009). Although transcription of the interviews were outsourced to a professional transcribe, the researcher performed a final check on the accuracy of each transcription.

“Qualitative analysis entails segmenting and reassembling the data in the light of the problem statement” (Boeije, 2010, p. 93). Data in qualitative analysis “are sorted, named, categori[s]ed and connected, and all these activities entail interpretation” (Boeije, 2010, p.94). When data is sorted into parts or phrases (themes and categories) with meaning, this process is called coding and is the basis for qualitative data analysis (Boeije, 2010).

Interview analysis in the current study focused on finding meaning through coding (Kvale &

Brinkmann, 2009). “Coding involves attaching one or more keywords to a text segment in order to permit later identification of a statement” (Kvale & Brinkmann, 2009, pp. 201-202). The coding for the current study was data driven and not concept driven, which means codes were assigned throughout and not selected from a pre-developed list (Kvale & Brinkmann, 2009).

The ATLAS software package assists with this coding of data and was the main analysis tool for the raw qualitative interview data. Boeije (2010) describes the following processes with which a software package can proof useful:

• filing of any electronic data in different folders;

• editing of all electronic data for better presentation;

• coding of data during open coding by the researcher;

• retrieving any electronic data or codes for an overview of all relevant data;

• searching for words or phrases within documents for frequency and place of occurrence;

• memos can be made within the software programme;

• visualisation of your data and codes can take the form of matrices and other graphics for visual

• the writing process is helped with ease of exporting data coding and themes to the report on the findings.

Memo keeping is also an important aid when doing data analysis. The process of identifying codes, categories and themes can be recorded systematically in the form of theoretical or analytical memos (Boeije, 2010). Memo taking involves recording any ideas that may related to data interpretation. Memos are a very good way to track logical progression in the research process. Memos were kept during the data analysis in this study and integrated into the final interpretations of the themes. The ATLAS software package allows the researcher to make memos whilst analysing different sections.

Boeije (2010) describes the first coding sequence (open coding) as an eight-step process. This process begins at reading the whole document and then re-reading each line for logical segments that contribute to overall meaning. The researcher then has to decide whether a segment fits the research question and if it does; the fragment is named/coded. Each relevant segment is coded throughout the whole document. Finally, codes are compared for possible overlapping (in which case they are coded identically) and the separate, relevant (to the research question) codes then make up the coding scheme. Codes can be either descriptive in nature, or may also be interpretive of the data (Boeije, 2010).

When open coding has been completed, axial coding can take place (Boeije, 2010). Axial coding uses the separate codes and then tries to integrate the data by finding new categories from related separate codes.

After reviewing if the data is represented by all the codes and whether very similar codes should not be merged, codes can now be investigated for similarities and differences. Main codes with sub-codes can now be grouped thematically to find which the dominant ‘themes’ in the data are. Main codes/themes should be clearly supported by depth of sub-codes (Boeije, 2010).

The process by which all the categories of data (from the open and axial coding of each separate data set) are finally compared and integrated in order to obtain a description of the research problem is called selective coding (Boeije, 2010). An integrated understanding of the data can yield answers to the following questions posed by Boeije (2010):

• Are there themes that have emerged from multiple data sets?

• What is the essence of all the participants’ contributions?

• How and are the themes that have emerged related?

• What should be included to understand each participant’s perspective?

When the codes and themes are integrated and then used to provide and explanation for a phenomenon, as well as any causal links in the data, the analytical technique is called “explanation building”

(Yin, 2009, p. 141). After applying the explanation building analysis to each case, this technique can also be extended across cases in a multiple-case study approach such as the current one. Here general explanations are sought that would explain and fit the findings from each of the cases, even when the detail from each case may differ (Yin, 2009). “Cross-case synthesis” still assumes the independence of each case, but then pools the findings from each case study into a “uniform framework” or table (Yin, 2009, p.156). For this

purpose, a table representing findings from all three cases will be related and discussed at the end of the case study results chapter.

3.4.2.2 Reliability and validity

Reliability refers to obtaining consistent results when replicating a measurement (Graziano & Raulin, 2000). In qualitative research, establishing reliability can be difficult. One cannot readily assume that a respondent would respond consistently to the same open-ended question on two separate occasions. One can, however, report on the exact interview procedure, interview schedules and data analyses methods in order to increase replicability of the research. This chapter serves to adhere to this reporting in order to increase reliability of the data.

Reliability in this study is also promoted by relating the transcription procedure and careful preparation of the interviewer (for example by avoiding leading questions) (Kvale & Brinkmann, 2009). The computer-aided data analysis process also increases reliability of results by providing exact references to where coded quotes can be found. This makes it easier for other researchers to follow the trial from quote to code to interpretation of the theme.

The accurate and true description of a phenomenon refers to validity of results. “In qualitative research, the term ‘validity’ is used in a rather differently defined meaning: the aim is to provide research with a guarantee that the report of description is correct” (Eriksson & Kovalainen, 2008, p. 292). Induction and reflexivity are terms associated with establishing validity for qualitative research (Eriksson &

Kovalainen, 2008). Induction refers to the process where coded data from the research cases (specific) are generalised to theory (general). Reflexivity refers to the continual evaluation of the researcher’s own influence on the research process from data collection to conclusions.

Reflexivity by the researcher represents the acknowledgement of the researcher’s own influences and part played in the research (Eriksson & Kovalainen, 2008). Research is inherently subjective, even when using so-called objective or empirical measures. Reflexive objectivity refers to the recognition that the researchers themselves are part of the research process and influence meaning making in an interview (Kvale

& Brinkmann, 2009). A researcher approaches any issue with his or her own background, preconceived ideas and prejudices. These should, at the very least, be recognised and, preferably, recorded for the research audience to contextualise a researcher’s interpretations of any given phenomenon.

An acknowledgement and reflection on the researcher’s own experiences in the topic may prove useful. I do not have a disability. This study is a continuation of my interest and exploration of the field of disability studies and positive psychology. My Master’s degree research has explored psychological well-being of visually impaired persons and the influence of a guide dog as more than a visual aid (Wiggett-Barnard & Steel, 2008). The current proposed research interest in PWDs in the open labour market stems from the my own experiences while working with the Wheelchair Users Forum of South Africa and through work in corporate social responsibility in recent years.

In some aspects, I could be considered a novice researcher (both in age and experience). This had the potential to influence the power dynamic when working with older and established business contacts. An

able-bodied, Caucasian female may also have caused reaction in the interview participants. Race, gender, age and disability status differences may have influenced the interview dynamics. Also, some perceived power differences may have to be recognised between a researcher/interviewer and interviewee, especially with workers in the lower skilled jobs.

In qualitative research, there is a diminishing of the distance between the researcher and the participant (Eriksson & Kovalainen, 2008). With personal contact, one assumes that the participant is the expert in the topic and that they can better inform the researcher. Participants have a unique perspective to give on the topic and increased familiarity can extract this voice. As the primary instrument in the research, through researcher’s speech and writing, the researcher recognises their role as an agent in the research (Eriksson &

Kovalainen, 2008).

There are seven stages in validating (called induction by Eriksson & Kovalainen, 2008) interview research (Kvale & Brinkmann, 2009). Thematising is the first step and refers to sound theoretical background which led to valid research questions. Designing refers to the study’s design and whether it’s an accurate and ethical choice for what and whom is under investigation. Interviewing validity refers to in-interviewing questioning and validation of an interviewee’s facts and meaning making. Transcribing method (verbatim or written text) should be valid for the type of interpretations that will be made from the data.

Analysing of the data should be validated by asking valid research questions that suit the data and also whether interpretations are founded. Validating refers to a researcher’s judgement call on which of the above stages is applicable to the current research and also on who will be approached to cross-validate findings.

Finally, reporting should be a valid reflection of the actual findings and also that the reader can play a part (called reflexivity by Eriksson & Kovalainen, 2008) in validating results (Kvale & Brinkmann, 2009, p.248-9). The researcher believes that these seven stages have been completed and makes the results of the case studies valid.

Lincoln and Guba (as cited in Eriksson & Kovalainen, 2008) replaced the more quantitative terms of validity and reliability with four other concepts for qualitative research criteria:

• dependability: research activities were logical and well defined;

• transferability: establish a theoretical connection between the current research and previous research;

• credibility: familiarity with data transcends to enough evidence to substantiate the research conclusions and/or interpretations;

• conformability: the actual data and conclusions have to be linked and understood by others.

Again, the researcher believes that the validity as explained by Lincoln and Guba (as cited in Eriksson

& Kovalainen, 2008) have been proven throughout this document.

Participants themselves can also help validate research data. When participants are involved with reviewing your interpretations based on their interviews, the researcher is performing a “member check”

(Eriksson & Kovalainen, 2008:293). All interview participants received a summary of the analysis of their

individual interviews. All participants had 10 days to respond with any corrections to the data. This served as both a member check and to satisfy ethical requirements.