4.5.1 Process
Hussey and Hussey (1997, p.74) explain that questionnaire survey results accompanied by interviews “provide qualitative insight and illuminations”. They comment that triangulation “can overcome the potential bias and sterility of a single-method approach”. Such a view is also supported by others (Saunders, Lewis & Thornhill, 2009) and a mixed method approach was considered appropriate for this research.
The interview method was used to add depth to the questionnaire responses and further explore the practices employed by institutions. However, it is recognised that interviews can be criticised on the grounds that interviewees may offer the views that they believe the interviewer wishes to hear rather than what they really believe (Horton, Macve & Struyven, 2004). It was hoped that assurances about confidentiality and the engagement of a sufficient number of interviewees would assist in countering this potential bias even if the possibility could not be eliminated.
The interviewees were selected on the basis of the information contained within the responses to the questionnaire, the need to cover a cross-section of the respondents of differing types and size of institution, and the likelihood of a favourable response to an interview request.
The length of the interviews varied between 35 minutes and 4 hours 45 minutes (the latter involving an interviewee with a strong interest in the subject matter), with an average time of 2 hours. The frequency of length is shown in Table 4.7 below:
97
Table 4.7 Time period for interviews
Time (minutes) 30-60 61-90 91-120 121-150 151-180 181-210 211-240 241-270 271-300
Number of interviews 9 7 11 5 3 3 1 2 1
Percentage of interviews 21% 17% 26% 12% 7% 7% 2% 5% 2%
The interviews commenced with an introduction to explain the research and the progress made to date before moving on to address specific questions. The first of these was a ‘breaking-the-ice’ enquiry to put the interviewee at ease and commence a discussion, such as asking about their background or the key stages in their budget cycle.
A total of 25 post-questionnaire interviews were conducted over a 14 month period, following an initial 17 interviews undertaken during the pilot stage (section 4.3.2). Table 4.8 below identifies the types of institution interviewed.
Table 4.8 Analysis of survey population, respondents and interviewees
Population Population Responses Responses Pilots Post-Survey
(Number) (%) (Number) (%) Interview Interview
Pre-1992 Universities 79 48.5% 40 47.6% 9 11
Post-1992 Universities 69 42.3% 40 47.6% 8 14
Colleges of Higher Education 15 9.2% 4 4.8% 0 0
Total 163 100.0% 84 100.0% 17 25
Whilst many researchers are in favour of recording interviews (Haynes & Mattimoe, 2004), no interview in this study was recorded. This was to ensure that the interviewee did not feel inhibited to respond freely to the questions raised. A number of pilot study respondents indicated that they would be uncomfortable having their views recorded and would be less willing to participate. This might partly be explained by the fact that the research was addressing an area containing commercially sensitive data, a view supported by the unwillingness of the Funding Councils to release forecasting data on individual institutions. The reluctance of interviewees to provide information in the presence of a recording device has been encountered in other studies (Bedard & Gendron, 2004). A view was taken that the potential loss in research rigour and validity was acceptable in order to gain a more detailed insight from interviewees. The possibility of secretly recording the interviews was rejected as unethical.
98
Notes were taken during the interview and interviewees generally recognised the need for brief pauses during the discussion for notes to be jotted down and allowed time for this. In order to ensure accuracy and comprehension of the interview, the notes were revised and reconstructed in a Word document within 24 hours. This allowed abbreviations and unclear handwriting to be converted in to a readable narrative whilst still fresh in the mind.
Although these notes reflect the discussion, they are not transcripts of the interview. In an attempt to counter loss of validity, interviewees were asked during the discussion to verify some of the notes that had been taken. Engaging an independent person to also attend the interview would have increased the validity of the notes but was not deemed to be practical. Interview notes were not sent to the interviewee for confirmation or signing-off as the material sometimes contained controversial or sensitive information. It could not be certain that the interviewee would be the only recipient of them or that they would not wish to withdraw what had been discussed having had time to reflect on their content.
A pre-interview set of questions covering certain themes, including items based on the interviewee’s questionnaire responses, was prepared in order to undertake a semi-structured investigation. Some questions could not be easily addressed by the questionnaire as they potentially involved a detailed answer, such as ‘How do you go about setting the University’s annual budget?’. Other questions followed logically from the analysis of the pilot questionnaire results. For example, many took a cautious approach to their budgeting. Interviewees were therefore asked ‘Is a favourable variance better than an adverse variance?’. Thus using qualitative data to complement the quantitative (Bryman & Bell, 2011).
This approach allowed interviewees some freedom in addressing enquiries on practices based on their expertise and experience, and to express their own opinions. Twenty one face-to-face interviews were held at the respondent’s own institution, with a further four being telephone interviews. Easterby-Smith, Thorpe and Lowe (1991) suggest that semi-structured interviews are appropriate where the subject matter deals with commercially sensitive data and the interviewee may be reluctant to reveal the truth about the issue other than in a confidential one-to-one discussion. This approach allows responses to be probed further.
99
The interview questions were slightly improved throughout the period of the interview stage based on the experience gained from previous interviews. Although there are arguments for the same set of questions to be asked of all interviewees (Yin, 1983), the counter to this is that such iterations are important as new insights emerge (Marginson, 2004). The information gathered from the interviews stage was not used for rigorous hypothesis testing and it was therefore considered appropriate to make improvements to the questions raised. An example of a standard set of questions is contained in Appendix IV.
Confidentiality assurances were given at the outset that comments would not be attributed to any individual or institution. Hence, in order to preserve anonymity, the origin of quotations has not been identified. This inevitably leads to some loss of transparency in the analysis undertaken. However, an identifier is used for each interviewee so that connections can be drawn between different points made by the same interviewee, and whether emphasis has been placed on particular interviewees. The identifier starts with either OU (old university – Pre-1992) or NU (new university – Post-1992) and is followed by a number which is specific to the institution interviewed.
The interviews provided a rich source of additional information to complement the details obtained from the questionnaire. They helped to explain some responses and to develop areas of understanding particularly in relation to efforts made to improve processes and procedures. Part of the difficulty of this stage of the research was deciding how many interviews should be undertaken. Marginson (2004) explains that the optimal number is at the point of ‘saturation’ whereby there is minimal incremental gain as similar responses are received at further interviews, but that it is also dependent upon time and resource constraints.
4.5.2 Method for analysing findings
An effective approach to analysing interview findings is to group them into themes (Bryman & Bell, 2011) and to integrate them with survey questionnaire findings (Creswell & Tashakkori, 2007). In order to do this the notes from the interviews were colour coded according to main topics based on an overall review of the information gathered. These topics addressed organisation structure, reporting processes and practices, carry forward of unspent budgets, liaison and business
100
partnering activity, resource allocation and contribution models, budget processes, scenario modelling, budgeting and forecasting accuracy, information technology and systems, approaches to medium term forecasting, student number forecasts, etc. This allowed the narrative to be constructed by reference to those colour coded groups. It was considered that the interview data was of a sufficiently small size to permit a manual review by theme rather than employing data analysis software such as NVivo to code the data prior to undertaking searches of a particular grouping. However, the allocation of interview data to a specific topic is an iterative and subjective process and it is recognised that there is a risk of human error by manually coding (Welsh, 2002). Nevertheless, a manual approach was adopted, taking care to allocate interview findings appropriately.
Creswell (2014) explains that the challenge for researchers is how to converge or merge quantitative and qualitative data. He offers three approaches. The first is to report on the quantitative results and then discuss the qualitative themes or alternatively vice versa (a side-by- side comparison). The second is to merge the two databases, changing the qualitative themes or codes into quantitative variables by counting them (data transformation). The third is to merge the databases into a tabular or graphical output which results in a single vision (a joint display of data).
For this study, a side-by-side approach is adopted. The interview narrative included in Chapter 5 follows-on from the survey questionnaire findings in order to add depth to those findings. The interview data, which was sub-divided by theme, is mapped to and incorporated within the appropriate section of the questionnaire findings rather than providing completely separate sections for questionnaire findings and themes from the interviews. However, the interviews provide a data source in their own right and it would be too structured to link this directly to each of the key points from the findings. The interview data was repeatedly revisited in an attempt to ensure that no significant element was excluded from the resulting data analysis.
Whereas the findings from the quantitative survey data are analysed by looking at the average responses, the qualitative data from the interviews considers items specific to institutions. It is therefore possible to pick up problems in the qualitative data that are not apparent from the quantitative average.
101