Entrevistas al personal encargado de las comisiones

3.2.2.1 Assessment and curriculum

Participants in this study were students studying in the upper-intermediate level of a pre-admissions program leading to diploma or degree studies at one of the Vietnamese campuses of an Australian- based international university. The second data set involved the students’ results following assessments at the end of the ten-week English course. This course forms part of the international educational institution’s English pathways program, which has recently undergone a comprehensive overhaul to accommodate a focus on task-centred learning following the communicative approach to teaching, and the curriculum framework is based on the four elements of tertiary study requirements, the course, the students and the assessment (RMIT English Worldwide 2011).

The English pathways program is what Hong-Nam and Leavell (2006) term an “IEP” (recall: “intensive English program”). Students in each course are required to complete four hours of daily classroom study for a total of two hundred hours per course. Classroom sizes vary between 12 and 20 students per class, with students encouraged to work in groups as often as possible. The IEP course is designed to not only improve the learners’ general English proficiency, or BICS (recall: “basic interpersonal communication skills”), but also their CALP (recall: “cognitive academic language proficiency”) in preparation for tertiary studies in English (Cummins 2008). The distinction between these two types of proficiency is discussed in section 2.2.

It is important to highlight the fact that LLS instruction is not a formal component of this IEP curriculum, although there are numerous opportunities for teachers throughout the programs to introduce and raise awareness of these strategies, either implicitly or explicitly. Classroom situations may arise where it is appropriate to highlight the benefits of specific LLSs, yet this instruction is at the teacher’s discretion. For example, some teachers may instruct students on how to skim or summarise

51

a text (cognitive strategies) to improve their reading accuracy, speed or comprehension, when such an opportunity arises in context in the classroom. In addition to not being taught explicitly, LLS use is, of course, also not formally or explicitly assessed at all.

3.2.2.2 Assessment principles and model

Assessments at all levels of this pathways academic English program follow rigorous quality assessment and are based on, among others, the following principles (RMIT English Worldwide 2011):

Assessment:

• is based on objectives and outcomes directly related to the courses and tertiary study;

• is valid and reliable;

• meets industry standards; and

• follows current L2 learning theories.

The assessment model is based on two integral components, namely, ongoing or formative assessment (OGA) and end-of-course or summative assessment (EOC) which comprises a student’s course result. For the purposes of this study, the students’ EOC and course total scores were used as an indication of their English proficiency.

3.2.2.3 Validity and reliability of assessments

There are a number of quality assurance measures in place to ensure the validity, reliability and security of tests. Trialling and analysis of tests are imperative parts of the assessment production process. Firstly, tests are written according to the assessment specifications and are based on course outcomes and course content. These are moderated, modified and trialled several times with both L1 English speakers and EFL/ESL learners. Following the second pilot with a minimum of 50 students, scores are collected, analysed and compared with a benchmark test. The benchmark correlation is determined by Pearson’s r coefficient which measures the correlation between two continuous variables (Brown and Rodgers 2002: 170; and RMIT English Worldwide 2011: 8). Table 3.3 provides a summary of internal test standards. Reliability is measured by Cronbach’s alpha, while an item

52

discrimination measure of larger than 0.4 means a student with a high overall test score is likely to provide the correct answer for that specific item (RMIT English Worldwide 2011: 9). The lower the p score, the more difficult the item is, and vice versa. Unfortunately, these analyses have only been run on the reading assessments and certain versions of the listening tests to date.

Table 3.3 Internal test standards set for test characteristics (RMIT English Worldwide 2011: 8)

Measure Desired score

Cronbach’s Alpha > 0.8

Item discrimination > 0.4

Difficulty 0.4 < p < 0.9

Benchmark correlation > 0.3

Sample size Min. 30; ideal > 80

3.2.2.4 Conducting, weighting and marking of assessments

The OGAs are conducted in weeks four or five of the ten-week course and incorporate all of the four macro skills of reading, writing, listening and speaking. Similarly, the EOCs test all four skills based on the course curriculum, and occur in the final week of the course. Unfortunately, as all the tests used in this study are still active and used with current students, copies of the assessments are not included as an appendix as this could jeopardise the security and confidentiality of the exams.

Both types of assessments are paper and pencil tests, except for the speaking assessment, and are administered under examination conditions (RMIT English Worldwide 2011: 5). (See Appendix C for a summary of the assessments for the upper-intermediate course.) Listening and reading tests are marked according to a provided answer sheet by the class teacher, and double-marked by another examiner if a student fails, in other words, scores under 60%. The writing and speaking tests are assessed by two independent examiners according to established criteria (see Appendix E). Similarly, the speaking exam is assessed by one or two individual examiners, and recorded. If there are discrepancies regarding marks with speaking or writing, or for any reason there is a need for another, independent examiner, an internally-accredited key assessor scores the test a third time. Scores are entered into class result Excel spreadsheets, after which they are checked carefully by level leaders and the assessment coordinator, before being filed, and the results are released to students via

53

email. To pass the upper-intermediate course, students need an overall score of more than 60%, and cannot score below 60% on any of the EOCs. These examinations are very important for the students, as passing them can ensure entry into a concurrent English and diploma studies program, or the possibility to continue along the IEP pathway.