Consideraciones toxicológicas

Collocation knowledge has been described as “one of the most important types of ‘contextualized’ word knowledge” (Schmitt, 2010, p. 229). It is therefore understandable that the field has seen an increased interest in depth measures featuring this component in recent years.

Early attempts to measure collocational knowledge employed translation formats, such as in Bahns and Eldaw’s (1993) study of the English collocation knowledge of German speakers. They used German prompt sentences containing translation equivalents of 15 English verb+noun collocations and asked for their translation into English. The method, however, lacks tight control as translated sentences might be acceptable but not contain the targeted collocation. Other researchers have attempted to measure this kind of knowledge using cloze items (Farghal & Obiedat, 1995), which suffered from similar problems in that an acceptable but not the targeted collocation

65

could be inserted by the candidate. This issue of lacking restriction becomes even more salient when the entire collocation is asked for rather than just one element of the collocation. The inclusion of initial letters in the gaps has been suggested to constrain the choices of the test takers as well as the provision of L1 translations. The latter, however, is only feasible if there is no identical collocation in the L1 so that it is really the collocational knowledge in the L2 that is being measured. Gyllstad (2007) further criticizes the sample sizes, the unprincipled sampling and lacking reliability evidence in these early studies. Schmitt (Schmitt, 1998a) used a sentence elicitation task to measure productive collocation knowledge in his research. Candidates were asked to provide three sentences per target word, each constrained by a topical context. However, he reports that his scoring criteria were probably too generous and lenient and that this format is therefore limited in its usefulness.

More recently, Bonk (2001) has investigated several collocation test formats. He also used sentence cloze items but focused on the insertion of elements of either verb+object or verb+preposition collocations. As a third measure he used a four-option multiple-choice format but in an odd-one-out design, whereby three options contained valid collocations and the candidates were asked to identify the one option which contained an incorrect collocational usage. He administered the three tests to 98 Asian EFL students and found satisfactory reliability values for all measures except the verb+preposition cloze test. The population performed similarly on all three measures and high collocation scores correlated with high scores on a test of general proficiency. Based on his IRT analysis he concludes that the verb+object cloze measure and the multiple choice format work well. The findings also suggest that collocation knowledge might be an indicator of advanced proficiency or advanced word knowledge.

In a more decontextualized approach, Mochizuki (2002) presented 54 Japanese test takers with node words and four collocation options to choose from. The findings from his study, however, render the instrument questionable as reliability values varied considerably, probably due to the

66

homogeneity of the population. Also, the test items were not able to capture a meaningful gain in collocation knowledge after 75 hours of instruction. Barfield (2003), adopting a developmental approach reminiscent of the VKS, designed a scale by means of which students can be asked to judge their familiarity with a particular decontextualized collocation and its frequency. In addition, his test contained non-collocations to prevent guessing. The measure yielded acceptable reliability values and discriminated well between stronger and weaker groups of test takers. Even though Gyllstad (2005) problematizes that the test contains possible non-collocations, the scale’s focus on frequency should make it clear that typicality or probability rather than possibility of collocations is the appropriate criterion. Therefore, his critique does not seem justified in this respect. Much more problematic seems the scale itself, which is again a self-evaluation tool. Schmitt (2010) raises the point that the sampling of the targets lacks clarification. According to him, a rating of the highest interval is only appropriate if the target collocation is indeed highly frequent. It is, however, not clear whether all real collocations were highly frequent or how the collocations were assigned the appropriate interval if they were not (Schmitt, 2010).

Gyllstad (2005, 2007, 2009) attempted to develop two collocation test formats that focus on receptive collocation knowledge and also include the frequent category of delexical verbs. His format COLLEX 5 asks candidates to select the real verb+noun collocation from three options. The 50 test items were controlled for frequency of their components and the targets had to feature a minimum z-score of >3. A corpus analysis was used to check that the distractors were not real collocations.

Gyllstad’s alternative format COLLMATCH 3, initially developed as a multiple matching grid, resembles the checklist yes/no format in its latest version. Candidates simply indicate whether a presented verb+noun combination is a collocation or not. While the initial matching format suffered from several shortcomings and was rightly discarded after the piloting, the current format of COLLMATCH seems not only misleadingly named but also prone to the same

67

problems of traditional yes/no tests. Despite reasonable reliability values and the ability to profile clear progressions along proficiency levels of both formats, COLLEX 5 seems a much more solid and meaningful way of assessing collocation knowledge.

Eyckmans’ (2009) Discriminating Collocations Test (DISCO) basically also follows the principle of a yes/no format with interspersed non-collocations. However, her longitudinal validation study is based on the results of only 25 students, which limits the meaningfulness of the investigation.

Revier (2009) criticises previous collocation knowledge assessments’ (e.g. Gyllstad, 2009) reliance on “a single elicitation method that involves presenting test takers with a node-word prompt (e.g. attention) and asking them to select or supply one or more collocates (e.g. call, draw, pay) of that node word” (p. 125). This, he claims, only gives little or no insight into the candidate’s knowledge of the whole collocation (Revier, 2009). However, knowledge of the whole collocation might be a desirable target for assessment, particularly for collocations that function as phrasemes rather than simply partners (Macis, 2013). Revier thus strongly argues for a relativisation of Nation’s (2001) view of collocation as a word-property or subcomponent of word knowledge. According to Revier (2009), collocation knowledge should be viewed as independent knowledge, whereby collocations are treated as meaning units in themselves, which is why he calls for assessments that require the candidates to produce or recognize whole collocations.

He therefore suggests CONTRIX, a matrix format in which test takers construct the collocation to fill a sentence gap from potential constituents. The format provides some context and restricts responses as learners select from a limited number of choices. This provides a reasonable alternative to cloze gaps that indicate the initial letters of the targets. The 45 item test was balanced for semantic categories, verb constituency, item frequency and noun-constituent frequency. It was piloted on 56 Danish EFL learners and showed promising psychometric results. However, Revier (2009) admits that a number of individual items performed poorly and are in need of revision. Further

68

validation evidence of an improved test appears necessary, although the format certainly bears potential. However, it seems debatable whether the test really taps productive collocation knowledge, as claimed by the author (Revier, 2009).

2.3.3. Test batteries assessing more than one aspect of word