Para el Objetivo Específico 2: OE2, se tiene:

The next research question was developed to seek backing for the third assumption supporting the explanation inference. This assumption states that the more frequent a collocation is, the easier the corresponding item will be for test-takers. This relationship is generally assumed to be true, based on the theory of frequency discussed in chapter 3. The theory is that test-takers should perform better on test items that are based on frequent collocations. The interpretation of this notion could be related to the item difficulty.

Theoretically, items that contain frequent collocations are easier for the test-takers than items with less frequent collocations. Backing for this assumption will support the underlying warrant that expected scores are attributed to a construct of collocational ability in academic writing, which includes knowledge of whole collocations.

This research question investigates whether item facility, the difficulty of an item, is related to the frequency of occurrence of a collocation in the target domain. In other words, are more frequent collocations easier for test-takers? In order to answer RQ 3, correlations were calculated between the item means for each scoring method and the rank order of the collocations, from most frequent to least frequent. Table 5.6 shows the rank order of the collocations by descending frequency, as identified in the corpus of general written academic English. The list shows collocations consisting of the verb, which is the collocate, and the noun, which is the node, followed by the form of the collocate used on the test in

parentheses. The list is sorted by raw frequency of the collocations in the corpus of written academic English.

Table 5.6. Rank order of collocations by frequency in the corpus of written academic English

Rank order Collocation Raw frequency Rank order Collocation Raw frequency

35 Have an effect (have) 2484 17 Give notice (given) 276

34 Make a decision (made) 1175 16 Make reference (made) 255

33 Play a part (play) 801 15 obtain information (obtained) 252

32 Provide service (provided) 682 14 Make effort (made) 241

31 Have control (has) 629 13 Have understanding (have) 239

30 Make an attempt (made) 551 12 Make contract (made) 234

29 Pay attention (paid) 503 11 Give attention (given) 229

28 Use technique (using) 481 10 Have chance (have) 217

27 Use term (used) 473 9 Make a choice (made) 212

26 Have access (have) 464 8 Reach a conclusion (reached) 208

25 Take step (taken) 452 7 Make difference (make) 199

24 Take care (taken) 413 6 Obtain results (obtained) 164

23 Take form (take) 412 5 Give consideration (given) 164

22 Take action (take) 404 4 Make statement (made) 163

21 Make distinction (made) 351 3 Receive attention (received) 151

20 Make contribution (make) 346 2 Make arrangements (made) 141

19 Treat patient (treated) 329 1 Exert an influence (exert) 89

The raw frequency count in the corpus for each collocation is also shown in Table 5.6. The range of raw frequency is from 89 to 2484 hits in 16 million words. Normalized values range from 5.5 to 154.5 per million words. These frequency counts were used to develop the test based on the notion of item development using high-frequency test items. Since collocations are not as frequent as individual words, the raw frequency counts may appear to be low, although the sample of collocations comes from the most frequent word pairs identified by the software analysis. Many collocations with lower frequency counts were not selected for the test because they were so infrequent relative to the other

collocations identified in the target domain.

5.4.1 Correlation between rank order and item facility with dichotomous scoring

A correlation was calculated between the rank order of the collocations and the item means of the scores using the dichotomous scoring method. Because of the rank-ordered data involved, a Spearman‘s Rho correlation was carried out on rank order of the collocations from the corpus and item facility of the same collocations on the test. The results indicated almost no relationship between the relative frequency of a collocation, as a word pair, and the item facility of the corresponding items on the test (rs = .02) using the dichotomous scoring method. This correlation between rank order of rank order and item facility is shown in the scatterplot in Figure 5.5

Figure 5.5. Scatterplot showing the relationship between rank order for collocation frequency and item facility using the dichotomous scoring method (rs = .02)

The correlation between rank order of items based on frequency and item facility for the test scores have shown almost no relationship. These results may indicate that frequency does not play a large role in item facility relative to the rank order in the corpus. This

interpretation is based on rank order of collocations from a sample which are all considered highly frequent collocations. The correlational coefficient was thus an indication of

differences in item facility on a very small scale, which stopped at 89 occurrences.

Collocations could have been sampled with low frequencies down to about five occurrences in the entire 16-million-word corpus. The results might have been different by including collocations with a larger range of item frequency; however, the range in frequency for the sampled collocations was limited to the group of highly frequent collocations, following the specification of the test calling for items to be developed using high-frequency collocations.

0 5 10 15 20 25 30 35 40 0 10 20 30 40 R an k Or d e r IF Dichotomous Data

5.4.2 Correlation between item frequency and item facility with polytomous scoring

In this section, the relationship between rank order of collocations by frequency and item facility with the polytomous scoring method is presented. Because of the ordinal data using rank order of item frequency, the Spearman Rho correlation coefficient was used to calculate the relationship. The correlation coefficient indicated a moderate yet significant relationship between frequency and facility (rs = .40). A scatterplot showing the relationship between rank order for collocation frequency and item facility using the polytomous scoring method is presented in Figure 5.6.

Figure 5.6. Scatterplot showing the relationship between rank order for collocation frequency and item facility using the polytomous scoring method (rs = .40)

The correlation between rank order and item means was stronger using the

polytomous scoring method than the dichotomous method. This scoring scale appeared to be more sensitive to the rank order of sampled collocations from the corpus. Even with the smaller range in frequency, a relationship begins to be evident between item facility and item frequency with the polytomous scale. Furthermore, Williams‘ test was used to test the null

0 5 10 15 20 25 30 35 40 0 10 20 30 40 R an k Or d e r IF Polytomous Data

hypothesis for correlations with dependent correlations (Steiger, 1980). Results indicated that correlation coefficients for dichotomous and polytomous scoring methods were significantly different.