Modos de conexión de baterías

Open-ended essay, multiple choice and dictation are the tests which the researcher designed in order to investigate the patterns of writing that lead English-speaking learners to make categorised or undesignated errors. Since this study is triangulated, it aims, using these tests, to answer the thesis question. Whereas the tests are intended to unveil what errors learners make, other tools (i.e. questionnaire and interviews) seek to touch the background to disclose how these errors are made and may explain why they would occur in this specific context. Generally, the tests were designed to reflect the real situation in L2WS Arabic classrooms. The purposes of each test, and their design and structure are outlined below.

4.7.1.1 Open-Ended Essay Test (OEET)

As its name implies, the open-ended essay test (OEET) is intended to replicate reality. Although writing outside exam halls is not timed or scored, an open-ended essay is the nearest type of writing to imitate individual writing. Imitative writing, which is the fir st of the four types of writing performance, can afford a revealing assessment of the basic tasks of writing letters, words and limited sentences (Brown, 2004a). Open-ended writing test has to be direct and on general topics (Weir, 1993), but could be set in tight or loose settings with regard to timing, text length and compulsory or optional topics. What is meant here by open-ended essay test though is a test which allows test-takers to write on a very general topic, but within a limited time and length. Learners are however free in terms of planning, organising, and structuring their essays.

This move of requiring extended writing to measure writing ability, according to Alderson and Banerjee (2002), began in the late 1970s and remains in use in many language tests, such as IELTS and TOEFL. To an examiner, setting this type of test tends to be quite easy and direct, as many researchers observed (e.g. Weir, 1993; Hughes, 2003; Weir, 2005). This easiness and directness, on the other hand, raises concerns about the

97 test implications as well as its validity. Questions are being asked, for example, about the load on test-takers’ backgrounds, which they constantly draw upon in order to compose on such general topics (Weir 1993). In addition, researchers argue that students can accurately write sentences in the sense of syntax while they cannot produce appropriate text with regard to coherence (Hyland, 2002). Scoring this kind of test, among other criteria, is also debatable (e.g. task purpose, test instructions) in terms of validity. However, what matters here is that this open-ended test is used to elicit learners’ ability of writing Arabic as L2WS. The researcher is interested in discovering common orthographic errors which English-speaking learners habitually make while in writing Arabic. Hence, the learners' thoughts, creativity, coherence and even their grammatical errors will not be assessed due to the fact that this is out of the research question. Accordingly, those debatable concerns are relatively irrelevant.

 Test aim

The OEET was purposely designed to discover orthographic errors in productive writing which could be considered to be ‘common’ and yet would not appear in other indirect tests (i.e. multiple-choices and dictation) for one reason or another (e.g. individual difficulty, rare vocabulary). As the other tests used in this research were designed indirectly using existing data, this test may show other issues that have not emerged previously or were not shown in the existent limited literature. Moreover, this test accounts for the approach of examining the interference between the two writing systems, as this study is concerned particularly with English-speaking writers of Arabic as L2.

 How the test was designed

To ensure that the test is valid and that it fulfils its purpose as explicated above, it was designed in a careful way. Written instructions were given both in their L1WS and L2WS to the test-takers on how to answer the question. It is believed that doing so makes the test more valid and reliable (Atkinson, 1987; Macaro, 2005).

The instructions include a time limit, acceptable length of text, and the task topic and type (see appendix 2). The acceptable length ranges between 250-300 words, which suits the purpose, and is neither too long nor too short. The time limit was set to 35 minutes on the

98 basis that one minute would be enough to compose 10 words; therefore 35 minutes is sufficient for writing up to 300 words and to review the whole text. The test entails one type of text, which is short essay, and one topic only (‘the last summer vacation’), for two reasons: restricting test-takers’ choices, which limits probable confusion or hesitancy; and maintaining comparable production of texts. Granting candidates more freedom in an already free, direct and open-ended test could cause invalidity and unusable results. The topic selected, namely writing on a past holiday, presumably fits all candidates. That is to ensure that relying on the test-takers’ backgrounds would not be difficult, and therefore individual differences would be pretty limited.

4.7.1.2 Multiple-Choice Test (MCT)

Although it is a traditional method and comparatively hard to design (Alderson and Banerjee 2002), unlike open-ended test, MCT is a type of test that is more desirable due to several attributes. Being indirect, showing clear purposes, and enclosing fixed response format are some of those attributes. One of the foremost types of indirect tests, as McNamara (2000) illustrated, is the Multiple-Choice Test. These characteristics in general prepare for a controlled test environment and assist raising validity in both test stages: testing and scoring.

Apart from the advantage of being indirect and practical, MCT can be scored quite quickly and reliably (Brown, 2004a). Its advantages, however, do not negate the fact that it requires certain knowledge of vocabulary for test-takers to react independently (Alderson and Banerjee 2002), besides that it is difficult to design perfect questions for an optimum test (Hughes 2003). Designing such a test requires writing several plausible distracters, in which ‘several’ means, at least two, and ‘plausible’ indicates the ability of each distractor to be possible but not correct. On top of that, each question, including its distracters, has to be clear, simple, and by no means tricky (Haladyna et al., 2002; Brown, 2004b). These criteria are supposed to limit the chance factor while maintaining unbiased and acceptable presentation. Out of different forms of MCT (e.g. one correct answer, two correct answers and True or False), the ‘one correct answer’ has been chosen for this study, for limiting the guesswork.

99  Test aim

MCT is utilised by this study to examine the existent data of common errors collected during piloting of the study, besides what previous studies have found and identified. The test would also allow the researcher to compare its results as hypothesised in literature against the results of the open-ended test.

 How the test was designed

The researcher designed the test following the criteria for writing a MCT. Since it looks for errors in L2 writing, the focus was on orthographic issues in word form, such as errors in writing graphemes, letter shapes, as well as orthographic and phonological errors. In addition, issues concerning cross-linguistics were included, such as directionality and transferring errors. The test was built based on the results of the pilot study. The distracters were made of actual errors that the students made while writing in Arabic. The test had 30 questions. Each of which had three options; obviously only one was correct (see Appendix 3). Each question/stimulus was written to test only one possible error. Each stimulus used in the test was incorporated in a sentence to help the test-taker recognise the correct meaningful word from the context – avoiding homographic words. The stimuli were divided into thirteen categories according to the error categories found in pilot study as well as in OEET. They were distributed based on their frequencies of occurrence as appeared in the results of the pilot study, OEET, and in the available literature. Hence, six stimuli involved letter shape, five entailed orthographic errors, four were given to phonological problems, and only one was concerned with directionality. The distractors were handwritten, imitating students’ errors as they appeared in their actual writing (e.g. pilot study, student essays). The time limit was set according to the numbers of questions, with each question being given one minute, which makes 30 minutes adequate.

4.7.1.3 Dictation

In the learning context, dictation is the process of writing down what the learner has heard, as he/she transfers the language from a spoken to a written form (Cartledge, 1968). It

100 involves, by this meaning, two language skills: listening and writing. It was recognised as a language exercise as well as a testing device. Although Cartledge (1968:227) argued that dictation ‘is not a teaching exercise but a testing exercise’, the use of dictation for testing purposes has been consistently exploited since the late 1960s (John et al., 1975). As it has been well-known, mainly for testing spelling, it has also been a useful tool to measure overall language proficiency, as John et al. (1975) points out. Even in L1WS, spelling was tested chiefly by dictation in order to identify learners’ visual recognition of letter sequence, and assess their explicitly or implicitly acquired ability (cf. Peters, 1985; Nunes and Bryant, 2006). In a dictation session, learners need to spell, contextualize and discriminate (Cartledge 1968), which adds analytic elements to the writing process (John et al. 1975).

Although reliable research sustains confidence in this technique being a valid and effective test (Lado, 1960; John et al., 1975; Hollenbeck, 2002), studies argue that judging natural writing by a dictation task might be risky. For example, a recent study reported that Russian writers rely to an extreme degree on memorisation of complete orthographic forms, as opposed to the orthographic rule (Kapatsinski, 2010). Furthermore, while Alderson and Banerjee (2002) acknowledge that dictation is proven to be useful in measuring language proficiency, they claim that it is not ‘effective’ when the pronunciation and orthography of the target language have a very close relationship. The ‘very close relationship’ probably means the superiority of phonology as linguistic components of a language as compared to the language orthographic system. If that is true, many phonographic languages, by this meaning, would not consider dictation to be effective in testing language proficiency. Further research is vitally needed to verify this claim.

Nevertheless, dictation is only one of several instruments utilised by this study, and so it is not used unilaterally to judge test-takers’ writing. It is indeed employed here as a research instrument that is concerned with the outcomes of L2WS and not the process of learning it. That said, the test most likely would be a successful device, along with the other tests, to elicit common errors among the study’s participants and to discover the extent of their orthographic knowledge.

101  Test aims

This test aims mainly to explore the implications of writing Arabic on the grounds of listening to the text, as many phonological/orthographical issues depend on how the text is heard. Moreover, it may verify results of the other tests.

 How the test was designed

The test extensively drew upon the pilot-study’s data in producing the dictating text, as it included the most common errors that previously emerged from the test-takers. From the thirteen categories that embodied those common errors, the dictating text was formed and compiled. The whole text was then divided into 22 various sentences, each of which has its own meaning and context. Instructions were given in advance about the process of the test. These instructions entailed guidance to listening to 22 full sentences, each of which had its own number. In the test sheet (see appendix 4), the test-takers find 22 numbered boxes. The participants were asked to write what they heard in the right numbered box as they listened to a pre-recorded tape of the text twice. The entire process was given 50 minutes, on the basis that each sentence had an allowance of one minute to be dictated and repeated once more. This presumably ensured that the time was just enough for writing the sentences, checking, and correcting their possible mistakes before the end of the task.