Variabilidad de los SSWs

Questionnaire design, production, distribution and analysis are complex, making this research tool expensive and time-consuming, but I took all those factors into account. A survey and an online focus group were appropriate in this instance, because access to the research populations was comparatively easy. The use of an online survey reduced the cost of production and the time for distribution and return and the use of computer software for analysis reduced the costs and time used in analysis.

The starting point of a questionnaire is the “transformation of general educational research and policy concerns into specific research questions for which the data are intended to supply an answer” (Siniscalco & Auriat, 2005: 5). The research questions (arising out of the objectives stated in Chapter 1) that this study addresses are:

 What mechanisms are in place in the South African education and training system to instil a quality assurance mindset or culture into all the various stakeholders? This has been answered in Chapter 2;

 Why has quality assurance become such a critical focus in education? This has largely been answered in Chapter 3;

 How do private providers perceive quality assurance in their respective organisations?  What problems are there with existing quality assurance policies, practices and processes?

and

 Are there any feasible alternatives to the current models and paradigms of quality assurance in private education and training in South Africa?

In order to get answers to these questions, I used three standardised questionnaires aimed at different target groups. A questionnaire is said to be standardised when “each respondent is to be exposed to the same questions and the same system of coding responses” (Siniscalco & Auriat, 2005: 3). The questionnaires used here contained a mixture of quantitative and qualitative questions with the inclusion of some questions requiring a ranking of opinion (or Likert scale), more typical of a quantitative survey, as well as space for the respondent’s reasons or own opinions to allow for qualitative analysis.

4.8.1 The Construction of the Questions

The questions followed the terms of Siniscalco and Auriat’s (2005: 22) classification, namely closed, open-ended, and contingency questions. The advantages of this approach are that it preserves the possibility of easy computation whilst providing respondents with the opportunity to present their own ideas. This is important in phenomenographic studies where the researcher seeks conceptual responses by the participants (Penn-Edwards, 2011: 18). Closed questions make it easier to compare the views of one group with another (Gray, 2004: 195), and help to avoid differences in interpretation (Coldwell & Herbst, 2004: 51). Closed questions may use a frequency scale, an importance scale, or an agreement scale (Siniscalco & Auriat, 2005: 23); the questionnaires used all these scales. A Likert scale was used to measure different views/opinions on the questions being posed. Closed questions were sometimes phrased as statements, for example, “Quality assurance is a critical business imperative”, and responses were noted.

Open questions have a potential to elicit responses unanticipated by the researcher (Gray, 2004: 194). Questions and statements included: “What suggestions do you have for improvements in the quality assurance system?” or “What are the key problems you have had to deal with in accrediting providers?”

The questionnaires by design asked closed questions with an opportunity for participants to comment on any aspect of the question, thereby adding an “open” element to the question. Where participants gave specific answers to a question, as in the Yes/No questions, they were also asked to clarify their answer in an open-ended follow-up question. The open elements allowed for qualitative analysis. The final question on all the questionnaires was a completely open question asking for the participants’ suggestions for possible alternatives to

existing quality assurance paradigms. This approach facilitated both quantitative and qualitative analysis.

Appendices E, F and G provide the questionnaires.

4.8.2. Dealing with Response Bias

Some of the closed questions asked about attitudes or opinions. The order of the answer choices provided in the questionnaire can influence the answers, however (Siniscalco & Auriat, 2005: 24). In constructing attitude scales, a vital consideration is changing the sequence of response categories and values to reduce response bias (ibid.). For example, if the choice of responses ranges from negative to positive or disagree to agree on a scale of 1 – 5, then the order on some questions can be reversed to range from positive to negative on a scale of 1 – 5 (ibid.).

Secondly, the questionnaire for tuition providers was pretested in a survey conducted during 2011 (Baumgardt, 2011: n.p.). Most of the questions did not present any difficulties, but I made some adaptations to the current questionnaire to allow for easier statistical analysis. The trial run asked participants to suggest improvements or make comments at the end of the questionnaire, and these were incorporated into the final questionnaire.

It would have been difficult to pilot the questionnaires for CEOs of professional bodies and ETQA accreditation managers because finding another group for this purpose in the small populations involved was impracticable. However, the Danish Evaluation Institute: Quality Procedures in European Higher Education NQA survey conducted in 2002 (as reported in European Network for Quality Assurance in Higher Education, 2003) provided guidance on the kinds of issues that should be addressed and the types of questions that should be used. The mixed-methods survey questionnaire using both closed (quantitative) and open (qualitative) questions thus supports the research paradigm underpinning the current research. Chapter 5 includes comments on problems with the questions.

4.8.3 Semi-Structured Interviews

Part of the qualitative research envisaged in this thesis used semi-structured interviews. These individual in-depth interviews allowed for a deep probing of the interviewees’ experience as key informants to accreditation and quality assurance of private providers (DiCicco-Bloom & Crabtree, 2006: 315).

They were selected as a data-gathering method in this thesis because of their flexibility. Although the essential questions were contained in an interview schedule, they were not set in a specific order as in a structured interview, and they lent themselves to deeper exploration of specific issues as the need arose (Merriam, 2009: 90). In order to explore an idea in greater depth, a question on the interview schedule may prompt further ad hoc questions as the interview proceeds (DiCicco-Bloom & Crabtree, 2006: 315). A semi-structured interview is not as loose as an unstructured interview, and tends to avoid discussion of unimportant and irrelevant issues.

Semi-structured interviews were conducted with one ETQA accreditation manager and one CEO of a professional body. An interview guide (Merriam, 2009: 114) provided a means of triangulation for the results. Each one-hour interview was audio-recorded for later transcription using the Livescribe Echo smartpen as an audio-recording device (it is unobtrusive and looks like a pen). This meant about four hours for transcription of each interview for purposes of analysis (IB Transcription Services, 2008: n.p.). The interviews took place in the offices of the interviewees, which contributed towards a more relaxed and natural environment (Boyce & Neale, 2006: 3).

Some cautionary issues concerning semi-structured interviews include the interviewees presenting a biased viewpoint, the time-intensive nature of interviews, lack of generalisability, and the skill of the interviewer (ibid.). The impact of these problems can be minimised by drafting an interview protocol (see Appendices B and C) and a good question schedule ahead of time, and scheduling the interviews for a defined duration. The generalisation problem is minimised by the results of the surveys. My experience in the field of accreditation of private providers meant I was able to guide the interviewees and keep them focused on the main issues.

4.8.4 Focus Group

Qualitative research also utilises focus groups. A focus group involves a group of people in conducting an interview, usually simultaneously and with a defined time constraint (DiCicco- Bloom & Crabtree, 2006: 315). I created an online focus group, and obtained answers in an asynchronous manner. According to De Wever, Schellens, Valcke, and Van Keer (2006: 2), this asynchronous arrangement is a common technique in educational research. Conducting the focus group online was advantageous as it allowed the discussion to take place over several days rather than having to be set up at a specific time and place. Participants had the

liberty to enter and leave the discussion at any time or to answer only some questions and not others.

The focus group consisted of members of the Skills Universe Forum, and more particularly the subgroup dealing with accreditation, assessment and materials development. In response to an online invitation, twenty-three people expressed their willingness to participate. The online invitation included the necessary informed consent request whereby members could indicate that they understood the focus group’s purpose, and what it would require of them. The group was closed and participation was by invitation only. The information was not accessible to the general public, as this is sometimes problematic with online forums. This dealt with the ethical problem of “harvesting” information without the contributors being aware of it (Stewart & Williams, 2005: 400).

A question schedule (Appendix D) was developed for the focus group, with the questions concentrating on the same issues dealt with in the questionnaires and interview schedules. A schedule ensured that the information was both pertinent to the research questions, and focused on the key issues of quality assurance. A total of ten questions asked over a period of several days allowed sufficient time for participants to respond, as well as discuss the questions in depth with me and the other participants. It was essentially an asynchronous, online conversation. Because the participants themselves typed their responses, no transcription from a tape-recording was required, as is the case for a viva voce focus group. This saved time. The transcript of this focus group discussion was used for qualitative data analysis and triangulation.

Merriam (2009: 108) suggests that the interviewer should be “distanced” from the focus group in order to avoid subjectivity and influencing focus group members to think in a particular way. This was pertinent since I am a member of the focus group’s progenitor group and I posed the initial questions from the question guide, and asked further probing or directive questions during the course of the online conversation. I avoided commenting myself, in order not to impose my thinking on the discussion.