This research study aims to explore whether academic skill development opportunities provided by the HEI in which I am employed make a positive contribution to online distance learning (ODL) students’ academic development. In the previous methodology chapter the data collection methods for this research study were critically discussed. The pilot results and key findings were included, together with the implications of those findings and the changes made to the online questionnaire as a consequence of utilising the pilot to test the questionnaire as a data collection tool. The online
questionnaire was used to gain an understanding of the academic skill development opportunities available to ODL students at the HEI at which I work, and students’ perception of the contribution these opportunities make to their academic development and in meeting their needs and expectations. In this chapter, the results of the strand 1 (QUAN) element of this research study will be presented and analysed, with the strand 2 (QUAL) data following in chapter 5. Integration of the pilot and strand 2 data will also occur in this chapter to highlight similarities or differences in the data.
A prime ethical consideration for this research study has been the maintenance of anonymity of participants. Therefore all participants will be identified, if necessary, via a pseudonym as follows:
Respondents to the strand 1 questionnaire will be identified as ‘Respondent’ and a number between 1 and 43 eg: Respondent 1 (R1). The respondent referred to in section 3.3.1 (chapter 3) who completed the online questionnaire twice will be identified by two numbers - R9/R29. It is not possible to establish with any certainty if this is the only duplicated response, but the data have been reviewed based on the assumption that no other duplication exists.
There were 3 incomplete responses to the strand 1 questionnaire which are included in the results. These will be identified as ‘Respondent Incomplete’ and a number eg: Respondent Incomplete 6 (RI6).
4.1 Presentation and analysis of the QUAN data
Strand 1 data were collected via an online questionnaire using Survey Monkey. Following application of the eligibility criteria and exclusion of students who were intermitting, responses were sought from 522 students. The three week deadline for completion of the online questionnaire elicited a response of 2.9% (n=15). Attempts were made to increase the response rate by sending further email requests to
participants (section 3.3.1 in chapter 3) and extending the completion deadline by two weeks. A further extension to the deadline was considered, but dismissed, in part to
76
avoid excessive emails which might have been perceived by students as intrusive or making additional demands on their time, but also to avoid any impact on the collection of the strand 2 data. The final response rate to the questionnaire was low (8%, n=43), but responses were received from students studying with all faculties and across the academic levels within the inclusion criteria.
Results are shown in the order by which questions were asked on the online questionnaire since these were designed to address the research questions7. By
presenting the results in this manner, answers to the research questions will begin to emerge and the students’ voice will remain at the heart of the analysis. Results are presented as descriptive statistics in the form of charts and frequency tables, as well as inferential statistics using the cross-tabulation chi-square test. The descriptive statistics outline baseline results for the study, providing background and context for the
inferential statistics which aim to confirm or disconfirm relationships or correlations between variables and testing of hypotheses. Descriptive and inferential statistics are presented together with the relevant online questionnaire question under discussion, as opposed to separate sections of the chapter, thus enabling rigorous analysis of key concepts and findings as they emerge from responses to the online questionnaire. These key findings will be revisited in chapter 6 when conclusions based on all the data are discussed.
Responses to questions 5 and 6 on the online questionnaire (Appendix 4) were measured as nominal data and provided answers to research questions 1 and 2. Although research questions 3, 4 and 5 were qualitatively driven, numeric data were collected via the online questionnaire. This took the form of nominal data to address research question 4 (question 8 on the online questionnaire), and ordinal data to measure students’ satisfaction with the academic skill development opportunities they had used and their perception of the contribution these had made to their academic development (questions 9 and 10 on the online questionnaire). The frequency of students’ use of academic skill development opportunities was measured via the collection of ordinal data (question 7 on the online questionnaire). This indirectly relates to research question 3, which seeks to establish when students access the different
7Research Questions
1. What academic skill development opportunities are available for ODL students? 2. What academic skill development opportunities do ODL students use?
3. When do students access the different opportunities available? 4. Why do students access academic skill development opportunities?
5. What are ODL students’ perceptions of the effectiveness of academic skill development opportunities in meeting their needs?
77
academic skill development opportunities available to them, data primarily collected via the online, audio-visual, semi-structured interviews.
4.2 Statistical tests
All statistical tests with the non-parametric data collected via the online questionnaire were undertaken using a combination of Statistical Program for the Social Sciences (SPSS) and Excel spread sheets. In all tests, no assumptions were made about the population, as is customary with non-parametric data (Cohen, Manion and Morrison, 2011). Descriptive statistics in the form of frequencies facilitated the organisation and reporting of the data and enabled observation of the profile of the data, but no
inferences or predications were made as a result of these descriptive statistics. Following this descriptive statistical analysis, a second stage of analysis looked for patterns, relationships or connections in the data (Denscombe, 2012). Observations are in effect a hypothesis, which can be defined as “a tentative explanation that
accounts for a set of facts and can be tested by further investigation” (Muijs, 2011, p.7). In order to test a hypothesis, a null hypothesis (H0) was created which stated there was
no difference or association between variables which was any greater or less than would be expected by chance (Heavey, 2015), and each null hypothesis is identified in this chapter as the analysis progresses.
Presenting a hypothesis in its null form places the burden on the researcher not to confirm that null hypothesis (Cohen, Manion and Morrison, 2011). As a researcher, greater interest lies in trying to establish whether a relationship does exist between variables and, therefore, attempt is conventionally made to reject the null hypothesis. Cohen, Manion and Morrison (2011) contend that ‘rejection’ of a null hypothesis is too absolute a term; claiming the strict parameters of research make it unlikely that rejection will be applicable in all cases. They argue for an alternative use of language, that of a null hypothesis being ‘supported’ or ‘not supported’; these are the terms used to present inferential statistics in this study. A second type of hypothesis, the alternative hypothesis (HA), states that a relationship between variables does exist; therefore, the
results in this chapter identify the alternative hypothesis when the null hypothesis is not supported.
An essential aspect of hypothesis testing is to establish the strength of any relationship between the variables. As such, it is essential to calculate the significance level, or probability value (ρ-value), since this provides an indication of the likelihood that an association between the variables exists. Statistical significance relates to the level of confidence that can be associated with the findings not merely being the product of chance, but does not automatically imply a level of importance in the findings. In the
78
social sciences the convention is to apply a significance level of ρ < 0.05, the interpretation of this being that no pattern or relationship is worthy of consideration unless the probability of them occurring by chance is less than 5% (Denscombe, 2012). Therefore, only patterns or relationships within this study which appeared to be
statistically significant (ρ < 0.05) led to a null hypothesis being not supported.
4.3 Chi-square test
Where interesting findings began to emerge from descriptive statistical analysis, hypothesis testing using the chi-square test (Χ2) provided opportunity to make
inferences and predictions based on the data gathered. The chi-square test is a flexible and commonly used statistical test which can be used with non-parametric nominal and ordinal data. The test measures the difference between the observed value and a statistically generated expected value (based upon the null hypothesis). Chi-square is a test of difference using univariate analysis and between two or more categorical
variables. Chi-square values are calculated via a cross-tabulation table, with the independent variable in the columns and dependent variable in the rows (Muijs, 2011). When using a 2 by 2 table in SPSS, a Yates’ Correction for Continuity is applied in the computed calculation which is designed to compensate for an overestimate of the chi- square value with a 2 by 2 table (Pallant, 2010). There are limitations to the chi-square test and these relate to the data in individual cells in the cross-tabulation table, namely: no cell should have an expected value of less than 1 and, no more than 20% of the cells should have expected value of less than 5.
The chi-square test introduces the notion of degrees of freedom (df) which refers to “the number of values that are free to vary when certain restrictions are placed on the data” (Blaikie, 2003, p.190). The number of degrees of freedom is related to the number of cells in a contingency table and in a chi-square test the degrees of freedom are equal to the number rows minus one times the number of columns minus one. Thus for a 2 by 2 contingency table:
df = (2-1) x (2-1) df = 1 x 1 = 1
In view of the small data set within this study, these issues did occur on occasions, in which case the chi-square test was performed manually instead of using SPSS. These manual calculations required the extraction of the relevant data from SPSS and
transferring it to a manual cross-tabulation table. Whilst this extraction of the data did introduce potential for error, this was minimised by rigorous checking. On a positive note, however, manual calculation of the chi-square test provided opportunity for
79
personal development by gaining a thorough understanding of the mathematical and statistical procedures which underpin the test.
4.4 The participants
Responses were received from students studying across the range of levels of course included within the sample (Figure 4.1), providing opportunity to gain an understanding about academic skill development from students with diverse entry requirements and educational experiences.
Figure 4.1: Respondents by level of course
Of the total number of respondents (n=43) most courses were delivered by FST8
(37.2%, n=16) or LAIBS (32.6%, n=14) (Figure 4.2). ALSS was only represented by one student response, but only one ALSS course (with 12 registered students) was included within the sample, possibly indicative of the low number of ODL courses delivered by that faculty. Interestingly, no students selected ‘FMS’, although closer inspection of the respondents’ data who volunteered to participate in strand 2 revealed that three were studying FMS courses, even though they selected ‘FHSCE’ on their questionnaire. This raises the question about how the change in structure of the faculties was communicated to students, or whether students are perhaps not interested in this level of detail and consequently had forgotten about their change in faculty name. Potentially of greater concern is the response from R30 who indicated
8 Faculty Names and Acronyms:
LAIBS – Lord Ashcroft International Business School ALSS – Arts, Law and Social Sciences
FMS – Faculty of Medical Science
FHSCE – Faculty of Health, Social Care and Education FST – Faculty of Science and Technology
0 5 10 15 20 25
Foundation Degree Bachelor Degree Masters' Degree
Number of respondents