There are a number of issues that can affect sample size in qualitative research; nonetheless, the guiding principle that helps to resolve this is the concept of saturation (Creswell 1998, Jette at al. 2003, and Mason 2010). Data saturation is the level at which new knowledge, new concepts and experience will not change the analysis away from the majority opinion or discovery (Creswell 1998). The issue about saturation is the ongoing debate on what constitute the adequate number for sample size saturation to be achieved in a study. Hence different authors have suggested what an ideal sample size for a qualitative study should be. Lee et al. (2002) argued that studies that either uses more than one method or conduct an in-depth interview require fewer participants as past findings supports quality of results for such studies. Again, Jette et al. (2003) suggested that having enough participants with the expertise in the subject under discussion can reduce the number of the sample size. What constitute “fewer” by Lee et al. and “enough participant” by Jette at al. remain a question as specific numbers were not given. It was Charmaz (2006) who recommended the use of a maximum of 25 participants for smaller qualitative projects whiles Ritchie et al. (2003:84) elucidated that qualitative studies should often “lie under” participants. Although these authors gave specific numbers, they did not offer evidence or reasons why a maximum of 25 or below 50 participants should be used. But Green & Thorogood (2004 & 2009) found that nothing new is found when transcribing after 20 participants are interviewed in most qualitative work and as such argued that participant of 20 should be an ideal maximum size when interviewing experts. Mason (2010) confirmed this after reviewing more than 500 PhD thesis and 300 articles and found that on average qualitative researchers’ use not more than 30 participants which supports Creswell suggested range of 20 and 30 participants (1998:128).
110| P a g e Just like most techniques that use qualitative approaches, there is no definite answer on what is considered an ideal sample size for expert judgement elicitation (Department of Energy 1984). Formal expert elicitation conducted on the choice of radioactive waste repository in the United States using data from past environmental assessment recommended the use of 500 sample size by selecting 100 experts from each of the five states involved (Department of Energy 1986, and Keeney & von Winterfeldt 1988). The final decision and reasons given for the choice of the site was found to lack relative importance to the cost of the project (Department of Energy 1986). The results from this exercise generated 42 lawsuits from residents living at the five selected sites contesting the reasons and facts that supported the findings. In contrast to the above, a review of five elicitation exercises with a maximum of 50 experts each from 2008 to 2010 helped in making informed decisions. These cases were: Terrorism litigation-hotel bombing scenario, statistical judgements-strengths of association in multiple scatter plots, Toxicology- chemical toxicity, HINI- federal responses to HINI pandemic, and Continuous Quality Improvement-features in health care (Dalal et al. 2011). A three round online based approach where participants were made to fill a “truth” question test and answer experience questions as part of the selection process from diverse group of broad experts. Again, using methods such as cluster analysis, data modelling, and analysis of variance/regression individual responses were analysed to derive a statistical aggregate. Results from all five cases received excellent ratings and commendation as each estimation recorded not less than 95% accuracy (Dalal et al. 2011).
Because the sample size for a research have an impact on the generalizability of the findings, it was recommended that depending on the availability of the experts and existing constraint a reasonable sample size that is not too large to contain irrelevant
111| P a g e data or too small lacking essential details is advisable(Rowe & Wright 1999). Some researchers maintained that depending on the desired sample size for the research problem at hand, the use of online approach to elicit expert opinion is suggested because of its ease to use, low cost factor and its ability to engage respondents at every stage of the process (Linstone 1978, and Rowe & Wright 1999).
Sunstein (2006) argued that since there is no “correct sample size” for research, focus should be on the objectivity, reliability and validity of the aggregate individual responses and so it is important to have less than 5 experts as a sample size that would provide credible responses than 500 full of biases and heuristics. Perminova et al. (2008) argued that the quality of an elicitation process is as good as the quality of the process of selection and the guidelines given to protect the sanctity of the results and not the sample size. On the contrary to the position of Sunstein (2006), if formal elicitation process is to be developed the issue about sample size cannot be disregard. Hence since the average project team size for offshore drilling is between 20-50 people across all the stakeholders, Azhar et al. (2008), Kaiser (2009), and Powell & Scyoc (2011) suggested that, the sample size of 20 should be the minimum for elicitation process for the oil and gas industry. The suggestion by the authors is consistent to the findings of Linstone (1978), Rowe & Wright (1999), and Dalal et al. (2011) on the need for an appropriate sample size and a well-structured question to minimise heuristics and biases as much as possible. This study would therefore use a minimum of 20 participants for the expert judgement elicitation in adherence to recommendations of earlier researchers. The next section discusses the choice of expert selection process.
112| P a g e