This definition of picture relevance is still rather vague if not further specified. This is of particular importance in the case of an assessment where responses have to be comparable. In order to avoid ambiguity in the analysis
process, the criteria of picture relevance had to be explicitly formulated. Consequently, the elements that were central to the interpretation of the picture were predetermined. These elements concerned two categories: the entities on one hand, and activities/properties on the other. The entities were the objects or persons (nouns) about which something was said and concerned the main figures in the pictures, often the agent. The activities/properties (verbs, adjectives, adverbs, and nouns) expressed the actions or described the entities. These entities and activities/properties collectively formed the minimal distinctive elements. A response was termed relevant if these minimal distinctive elements were present. Not all the listed entities and activities/properties were always required for relevance. Keeping in mind the complexity of the picture and the minimal distinctive elements required to describe the picture a balance between the two was made. Table 6:1 gives an overview of the scoring for the minimal distinctive elements.49
In Table 6:1 the relevance scores are given for responses requiring one, two, or three elements for either the entities or the activities/properties. In the category one element, if one element was given for a response, then the relevance score was relevant (R). On the other hand, if one element was required, but no relevant elements were given, then the score was not relevant (0). In the category two elements, if two elements were given then the relevance score was relevant (R). If only one of the two elements were given, then the relevance score was partially relevant (P), meaning that the response contained only some relevant elements – actually only half of the required number. This was judged to be barely enough to evidence any relevance. If no relevant elements were given, then the response was marked as being not relevant (0). For the pictures in the three elements category the scoring becomes a little more complex. If three elements were required and given, then the response was scored relevant (R). If two elements were given (two-thirds of the total number of required elements) this was sufficient relevance to be scored as relevant (R). If only one of the three required
49 The minimal distinctive elements for the picture description tasks and the picture story task are given in Appendix 2.
Table 6:1 Relevance scores and minimal distinctive elements (R = relevant, P = partially relevant, 0 = not relevant).
Response category One
element
Two elements
Three elements Required number of elements per
category 1 1 2 2 2 3 3 3 3
Given number of elements 1 0 2 1 0 3 2 1 0 Relevance score R 0 R P 0 R R 0 0
elements was given (one-third) then it was judged not relevant (0). Finally, if none of the three required elements were expressed, then the score was also not relevant. Summarizing, in determining the relevance score, the entities on one hand and the activities/properties on the other hand were counted. These were termed the elements of the picture. Depending if a picture requires one, two, or three elements, its relevance was scored. One score was obtained for the entities and one score for the activities/properties. The process for both was the same. This is summarized in Table 6:2. Subsequently, the scores of each of the two categories, entities and activities/properties, were combined producing the following total scores given in Table 6:2.
Table 6:2 Relevance scores for entities and activates/properties. (R = relevant, P = partially relevant, 0 = not relevant).
Scores of relevance
Score for entities R R R P P P 0 0 0
Scores for activities/properties R P 0 R P 0 R P 0 Relevance combined score R P P P P 0 P 0 0 In determining the minimal distinctive elements the focus was on the core activity portrayed in the picture. For example, Picture 3 in Task 6 (Figure 6:4) shows a man walking. The man is the entity. He is the main and only subject in this picture. This entity is clearly a man, shown by his baldness and mustache, thus it cannot be viewed as a woman. The activity that is depicted is lopen (to walk). A second possibility is wandelen (to stroll). No other interpretation can be given for this picture. A few details in some pictures can be interpreted in a number of ways, depending on the speaker’s perspective. For example, in the second series of pictures a more complex event is illustrated requiring a combination of entities or activities/properties. Picture 1 in Task 7 (Figure 6:4) shows a man getting a coat. Man (man) and
jas (coat) are core entities. The coat rack in this picture is not a core entity.
The core activity in this picture can be either pakken (to take), halen (to get) or ophangen (to hang up), depending on the point of view of the speaker. The last series of pictures for the picture description tasks were actual photographs of events. These were the most complex and could elicit various different responses. Picture 1 in Task 8 (Figure 6:4) shows a market scene where bread is being sold. This picture can be described from two different perspectives—that of the women buying the bread or that of the man selling the bread. The core entities from the first perspective are vrouw (woman) or
vrouwen (women), brood (bread) and markt (market). The activity could be
either kopen (to buy) or betalen (to pay) depending on which part of the activity is focused. Of these two activities ‘buying’ is seen as the core activity and ‘paying’ a subsidiary activity. Consequently, the activity
‘paying’ is marked as partially relevant. If the event is viewed from the man, the entities are man (man), brood (bread) and markt (market). The core activity is then verkopen (to sell).
To illustrate the operationalization of the scoring system the responses by two learners for three picture descriptions from the picture description task are discussed in detail in Figure 6:7. Each learner shows how she manipulates the language she has at her disposal to express relevance. The first learner, Yamina, has great difficulty formulating understandable utterances. Without the pictures, the listener would be unable to visualize what is happening. The second learner, Nadia, clearly has a larger vocabulary, which she is able to put to use. For the first picture Yamina does not express the core entity, in this case the agent, and the core activity, drink (drink). She is only able to say koffie (coffee), insufficient for a relevant response. Nadia is able to express the essence for both the entities and the activities. She even adds extra information by saying what the child may be drinking. For the second picture, only Nadia is able to convey what is exactly happening in the picture. Yamina seems to question her interpretation of the slightly bald character with a big mustache as a woman. Perhaps she expects a woman to be feeding a child, but this is an inference on the part of the researcher. Nevertheless, she does say “the baby eat” and for that her response is partially relevant. Nadia, in a short clear utterance, tells what the picture conveys. The third picture is the most troublesome for both learners. Yamina only expresses single words of items in the picture, but cannot connect them into a relevant description. Even her use of eten is questionable. In Dutch it can be the verb ‘to eat’ as well as the noun ‘food’. In Yamina’s response she could have meant either word. In any case, her response is not relevant. Nadia, in spite of some faulty grammar, uses her language knowledge optimally, clearly conveying what the picture depicts.
Task Yamina Nadia Minimal distinctive elements (P= partially relevant Task 6 O Koffie. (Coffee) R De jongen of meisje drinkt, misschien melk of water.
(The boy or girl drinks, maybe milk or water.) Kind /meisje/ jongen (child/girl/ boy) Drinken (to drink) Task 7 P Vrouw, vrouw? Baby eten. (Woman, woman? Baby eat.INF) R Man/opa + baby/kind (man/granddad + baby/child Eten geven /voeden/ eten = P
(to feed/to eat)
Man/opa + baby/kind (man/ granddad + baby/child Task 8 O Hond. Kijk 's. Koffie. Brood. Eten. Koffie. (Dog. Look. Coffee. Bread. Eat.INF. Coffee.) R Die vrouw, misschien zijn man of kind, zitten in de picknick. Gezellig voor de buiten zitten, gezellig eten. Misschien volgens mij naar buiten, bos.
(That woman, maybe his husband of child, sit in the picnic. Enjoyable for the outside sit, enjoyable eat. Maybe according to me to outside, woods.) Familie/ mensen/ man, vrouw, jongen + park/ picknick/ buiten (family/ people/man, woman, boy + park/picnic, outside) Picknicken/ eten en drinken (to picnic/ to eat and drink)
Figure 6:7 Relevance illustrated in three pictures from description tasks 6, 7, and 8 for two learners (R = relevant utterance, P = partially relevant utterance, 0 = not relevant utterance).
6.6.2 Criteria of coherence