The 25 individual items comprising this section of the questionnaire were developed from two constructs, Teaching Effectiveness (18 items) and Peer Group Interactions (7 items). They were each confirmed as valid and useful survey instruments from previous literature, as is discussed in greater detail below.
Both academic and non-academic determinants of overall satisfaction were considered when designing the Teaching Effectiveness construct. To measure student perceptions of course teaching quality, scales were drawn from the NSS, originally developed and piloted by Richardson et. al (2007), on which results were subsequently reported by Marsh and Cheng (2008) and, for science subjects, Fielding et al (2010). Marsh and Cheng’s (2008) purpose was to conduct factor analysis and multilevel modelling on the data from all participating HEIs over two years (2005 and 2006) of the 22-item UK’s National Student Survey (NSS). They confirmed six factors as intended measures for the original NSS, but suggested that a seven factor solution was most appropriate by splitting the Assessment and Feedback factor into two. They also indicated that although there was much variance in response between different universities, the large sample size within each institution meant that differences were stable and highly reliable over time (r=.86).
Six of these seven scales from the NSS survey, showing a moderate level of internal consistency as reported by Richardson et al (2007), formed the basis of the Teaching Effectiveness construct for this survey: Teaching (α=.79), Assessment (α=.75), Feedback (α=.75), Academic Support (α=.83), Learning Resources (α=.77), and Personal Development (α=.82). The Personal Development scale was the most recent addition to student surveys (Wilson et. al, 1997), in order to reflect the growing importance and focus on developing employable skills within higher education courses. The seventh ‘Organisation and Management’ scale from the NSS was not included, as it was considered to have less relationship with students’ teaching and support services quality perceptions, the main focus of the Academic System construct of this paper. A five point Likert rating scale, consistent with the design of previous instruments, was used to indicate level of agreement, from ‘strongly disagree’ through to ‘strongly agree’. (NB: the current 21-item NSS can be found on the Higher Education Funding Council for England’s website). The scales, some modified for this study, are presented in Table 3.3 below.
The Good Teaching scale contains four items which focus upon students’ perception of teaching quality, in terms of their interactions with teaching staff. These included the teachers’ enthusiasm and ability to explain, make the subject interesting and intellectually stimulating. A high score indicated good quality teaching as perceived by respondents.
Assessment: this sub-scale consists of two items based on clear marking criteria and fairness of assessment arrangements and marking, with a high score indicating positive perceptions in this area.
The Feedback scale contains three items pertaining to promptness, level of detail and clarification of feedback. A high score indicated effective feedback as perceived by respondents.
Table 3.3: Teaching Effectiveness Construct Items
Good Teaching Scale
4 Teachers are enthusiastic about what they are teaching 9 The course is intellectually stimulating
19 My teachers were very good at explaining things
22 The teaching staff did their best to make the subject interesting
Assessment Scale
8 Assessment arrangements and marking have been fair
12 Marking criteria have been made clear in advance
Feedback Scale
3 Feedback on my work has come back in time
7 I have received detailed comments on my coursework
16 Most feedback I received helped me to understand how to improve future work
Personal Development Scale
5 As a result of this course, I have more confidence in presenting my ideas 10 As a result of my course I feel confident about taking on unfamiliar problems 15 The course helped me gain a better understanding of people from different cultures 17 The course improved my skills in written communication
24 The course has developed my critical thinking skills
Learning Resources Scale
1 The library resources are good enough for my needs
21 I have been able to access general IT resources when I needed to
Academic Support Scale
6 Support has been available if I had extra learning needs (e.g. language, academic writing 13 I have been able to contact staff when I needed to
The Personal Development scale contains five items which aim to measure the extent to which the course has helped to develop the respondents’ skills, which, because they are generic and not discipline-specific, could be considered as important transferable skills for employment. NB: Although part of the NSS, and therefore kept in this part of the questionnaire, this scale could have been considered within the Institutional Outcomes dimension of the author’s conceptual framework, as it relates to the end result of students’ educational experiences: generic skills and abilities that students are expected to have developed upon graduation (Clemes et al 2008). For this study, two new items were added based on course learning outcomes and focus group results; an item on ‘critical thinking’ to reflect the vocabulary from focus group findings, and one on gaining an understanding of different cultural groups, which reflected a key skill emphasised in NBS Masters courses. The three original items relate to the development of written communication and self-confidence, and the ability to cope with unfamiliar problems. Again, a high score reflects a strong perception of improved personal development.
Learning Resources scale: As service quality factors outside of the classroom have also been deemed as important to students’ overall course experience (Richardson, 2005), the NSS’s 3-item scale (Richardson et al, 2007; Marsh and Cheng, 2008) (α = 0.77) which measured the adequacy of the library, access to IT resources, and access
to specialised equipment and facilities formed the basis of this construct. The latter item was not considered to be particularly relevant to this business school’s Masters courses, and thus was removed. A high score signifies a positive perception of the university support services provision.
Academic Support scale: Whilst effective library resources are an essential support service, Masters students, especially non-native students who did not study undergraduate degrees in a British higher education institution, may require extra study or academic skills support to produce the required standard of work expected at Masters level. In addition, it has been recognised that international students, particularly those to whom English is not their native language, also require extra support in the way of language. Therefore, Marsh & Cheng’s (2008) 3-item scale formed the basis of this scale, although only one item relating to ability to contact staff was unchanged. One item on making study choices was removed as it was more relevant to undergraduate courses, and one item on accessing advice and support was amended to accessing learning needs such as language support, which was developed from focus group findings and previous surveys (e.g. Kwan and Ng, 1999; Arambewela and Hall, 2009).
Peer-group Interactions (PGI) scale: The remaining seven questions provided in this section of the questionnaire relate to the Social System component of the conceptual framework. Although perception of academic quality has been found to be the most predictive measure of student satisfaction, other variables have been shown to play a significant role (Gibson, 2010). According to Pascarella and Terenzini (1980) a student’s integration into a HEI’s academic and social systems will affect academic adaptation and ultimately, performance. They report that student learning is affected by the extent to which peer groups are friendly, approachable and encouraging, concluding that for international students, a culturally diverse student population is more likely to influence “students’ sense of belonging and satisfaction with their college experiences and their commitment to their institutions” (Pascarella and Terenzini, 2005, p 419). This supports other research findings that, outside of the classroom, social integration and sense of belonging are important factors in students’ satisfaction with their overall course experience (e.g. Gibson, 2010; Arambewela and Hall, 2009). Therefore, Pascarella and Terenzini’s (1980) validated scale (α = .84) was used as the basis for this part of the construct (Table 3.4 below). The wording of the scale was slightly modified from focus group findings and previous literature, to better reflect the Masters courses with their culturally diverse student population. Three items, marked
by an asterisk, have been reverse coded. A high score indicates social adaptation and feelings of belongingness to the academic environment.
Table 3.4: Peer Group Interaction Scale (Pescarella & Terenzini, 2005)
11* I believe most NTU students have beliefs and attitudes different from my own 2 Since coming to NTU I have had close friendships with other students 14 The friendships I have made at NTU have been personally satisfying 18* I wish I had made friends with more students from different cultures to mine 20 As a result of this course I have made new friendships
23* I don’t know any students who would listen to me or help me if I had a personal problem 25 I have a social support network at NTU with whom I can share study and personal
experiences