The TOLD and Culatta’s narrative task are not the only instruments that present potential problems. Many other language assessment tests are subject to strong criticisms (Corrêa, Freitas & Lima, 2003). One of the most widely-used tests, the Clinical Evaluation of Language Fundamentals (CELF), both in its original and revised formats (Semel, Wiig & Secord, 1980, 1987, 1995), attracts similar criticisms to the TOLD. Problems with the CELF start with the very definition of the test’s aims. At the beginning of the revised version’s technical manual, the authors state that “CELF has proven to be a useful test for identifying students who lack the basic language skills which are the foundations of mature language use in communication: word meanings (semantics), sentence structure (syntax), and recall and retrieval (memory)” (my italics). As we will shortly see, like the TOLD, the CELF is not a test of basic language skills as it claims to be. The construction of the CELF was heavily based on Language Assessment & Intervention for the Learning Disabled (Wiig & Semel, 1980), written by two of the three authors of the test. An analysis of the book reveals major conceptual assumptions which are potentially problematic. As the book’s title indicates, there is no particular reference to SLI. This, in principle, does not seem to be a problem. Nevertheless, the terminology used by the authors to refer to their target population, along with passages that reveal what the authors effectively used as their ‘research object’, casts doubts on the adequacy of the CELF for identifying potential cases of SLI.
With respect to the target population of Wiig and Semel’s book, it is interesting to note that the authors make use of different expressions throughout the text. In most cases, ‘learning disabled children’ is used, but other terms are used with high frequency, such as ‘language and learning disabled student’,
Ա
紐
‘learning disabled adolescents’, ‘language disabled student’ and ‘language and learning disabled high school student’. Moreover, the authors often contrast the performance of ‘disabled children and adolescents’ with the performance of ‘their academically achieving age peers’. It is not clear, therefore, which is the target population of the study presented by Wiig and Semel. In addition, the authors make frequent use of terms such as ‘classroom’, ‘teachers’ and ‘school curriculum’, Can the terms ‘children’ and ‘students’ be used interchangeably in a study that serves as basis for a test which is supposedly assessing basic language skills? Should the ‘language’ formally taught in schools be the focus of such studies? I argue that those features greatly diminish the validity of the CELF as a measure of basic language skills, i.e. those mastered by any typically developing child by around the age of five, without any formal instruction7. The authors deliberately state that their “book seeks to put the day-to-day management of the learning disabled child with a language disorder squarely within the domain of the classroom teacher” (page vii) and that their focus was on the “language components of the curriculum D” (page vii). Such statements seem inconsistent with the attempt to use Chomsky’s generative theory, which, in 1960s’ and 1970s’ terminology, was concerned with the ‘intrinsic competence of the idealized native speaker’. In addition, as raised previously in this chapter, Wiig and Semel make inappropriate use of some of Chomsky’s terms, like many studies on language acquisition carried out within the developmental psychology framework of the time. Wiig and Semel start the section they called ‘Forming sentences’ stating that “in high school, grammar and English are the most difficult subjects for students with learning disabilities” (p. 60). They go on to say that there are many reasons for such difficulties, but “problems of memory and abstraction” would be of primary significance. Immediately after, in what they consider to be an attempt to better understand the difficulties encountered by high school students, they cite Chomsky’s transformational grammar (Chomsky, 1957), making reference to phrase structure rules and the concepts of deep and surface structures. The misinterpretation of the latter terms is particularly striking: “The surface structure reflects the syntactic properties of the sentence; the deep structure reflects the meaning. Thus, you could understand the surface structure of a sentence but not its deep structure, if you were unfamiliar with the words used; or you could understand the deep structure — know what ideas are being discussed — but not understand the surface structure — how the different words
7
The authors of the TOLD also use ‘students’ as well as ‘children’ in their manual, but not so often as the authors of the CELF.
Ա
苀
relate to each other” (Wiig & Semel, 1980: 62). To illustrate what they believe surface and deep structures mean, the authors provide the following examples: ‘Go to the library and return these books for me’, as supposedly the deep structure of ‘Take these books back’ (p. 62). The following quotes are also illustrative: “When we listen to a story, a lecture, or a discourse, we retain only the meaning or deep structure of sentences, paragraphs, and larger units” (p. 299); “when the youngster enters junior and senior high school D he must be accurate, efficient, mature and rapid in processing the surface structures of a variety of sentences and retrieving their underlying meaning” (p. 398); “to reduce the syntactic and transformational complexity of the language used in instruction, the teacher can use the guidelines we presented above for adapting reading materials” (p. 423) (my italics).
It is, thus, possible to say that Wiig and Semel’s (mis)use of Chomsky’s generative linguistics is purely as a descriptive tool, and not as a hypothesis about the nature of language acquisition. Indeed, no serious discussion about the nature of language acquisition or the relation between language and other cognitive abilities is undertaken by Wiig and Semel.
An additional problematic element is found in the CELF-R’s technical manual. It refers to the description of the reasons why the authors excluded from the revised version the subtests of pragmatics, which were present in the original version of the test. Among other reasons, Semel, Wiig and Secord say that the original version of the CELF was judged by most users to take too long to administer, which influenced their decision to drop the pragmatics subtests in the revision. Just like Tomblin’s statement cited above (see page 24), the decision of whether or not to include pragmatics in the test was taken (at least partially) not on the basis of theoretical and conceptual grounds, but on logistical grounds (duration of the test).
2.5.3.1 The CELF’s subtests
Like the TOLD, many of the CELF’s tasks do not tap basic language abilities, but skills which are not specific to language. The subtest oral directions, for example, in which the child is supposed to “interpret, recall, and execute oral commands of increasing length and complexity” (CELF-R Technical Manual, p. 8), is poorly defined. The concept of complexity is not explicitly presented, but does not appear to be pertinent to language development. Children are asked to follow
ऋ Ա
袐
instructions such as ‘Point to the first black triangle and the last small white circle’ or ‘Point to the last small black circle to the left of the big black square’. The reasoning behind the inclusion of such a task in a test which is supposedly assessing language abilities is questionable, as it demands skills which pertain to visual-spatial cognition as well as certain linguistic skills. A similar criticism applies to the subtest linguistic concepts, in which children are asked to point to different colour bars after hearing oral instructions. Questionable test items such as ‘After I point to a red line, you point to a blue line after you point to a yellow line’ and ‘If the red line is first, point to the yellow line’ are used with the children. The subtest sentence structure is similar to the TOLD’s grammatic understanding subtest in its design and format (picture selection task) and it is subject to some of the same criticisms I have proposed for the TOLD’s subtest, with respect to stimulus sentences and the selection of pictures. Many of the test items are difficult to represent pictorially, such as ‘The boy wanted to swim across the pool to sit with his friends’, ‘The woman asked: how much does this apple cost?’, ‘Father asked: shouldn’t you take out the rubbish?’ and ‘Mother asked: shouldn’t you play the piano now?’. Moreover, the two latter examples, which the authors classify as indirect requests, are of questionable validity. The interpretation of this sort of structure depends largely upon the context in which it is spoken and, therefore, involve extra-linguistic factors that should not be assessed (or at least should be avoided whenever possible) as part of a test which is supposedly evaluating basic language. Curiously, the book that provided the rationale for the construction of the CELF, Language Assessment & Intervention for Learning Disabled (Wiig & Semel, 1980), recognises the pragmatic complexity of indirect requests and the contextual and relational rules needed for their correct interpretation: “The pragmatic meanings elude them (language and learning disabled youngsters)” (p. 78). Oddly, thus, the authors provide — most likely unwittingly — arguments against the inclusion of indirect requests in a subtest assessing syntactic abilities.
Another problem of the subtest sentence structure lies in the design of the items with relative clauses. The four picture options for the test item ‘The man who is carrying his umbrella is walking out of the door’ do not provide the means for adequately testing the child’s knowledge. As figure 6 shows below, the pictures available for the child are: 1) man holding nothing and walking out of the door; 2) man holding umbrella and walking out of the door; 3) man holding nothing and a couple of meters away from the door; and 4) man holding nothing
Խ
슀
and walking out of the door (umbrella is open and laying by the door); The correct answer, undoubtedly, is picture no 2. However, the way the remaining options were designed does not allow the possibility of picking up an error that can potentially be made by the child. If the child misses out the relative pronoun ‘who’ and instead of registering ‘The man who is carrying his umbrella D’, registers ‘The man is carrying his umbrella D’, picture no 2 is still the most adequate answer. This test item, therefore, is not adequately manipulating the necessary elements to test the abilities involved in the comprehension of relative clauses.
Figure 6: pictures used on the CELF’s sentence structure subtest; test item is ‘The man who is carrying his umbrella is walking out of the door’
The lack of precision in the definition of what is really being tested in the CELF is reflected in other subtests as well. Further in the technical manual, the authors provide additional examples in which they in some way contradict themselves, rather as they did with the indirect requests subtest discussed above. Two interesting examples come from the subtest listening to paragraphs (in which children have to answer questions based on stories read to them) and the subtest word associations (in which children have to provide orally the maximum number of lexical items belonging to a specific category – such as animals or means of transport – within a time limit). At the same time that the authors claim that the CELF is a “useful test for identifying students who lack the basic language skills” (p. 1, my italics), they say that listening to paragraphs requires children “to recall details they have just heard (D) and to draw inferences based on those details” (p. 24). They go on to say that “Because several sentence boundaries are crossed before the first question must be answered, the subtest probes longer- term aspects of memory”. It can be argued that inference-drawing and remembering details such as the colours used by a character to paint a table in a
story (as is demanded in one of the test items) are not basic language skills and do not share the same cognitive mechanisms as basic language skills do. A similar argument can be put across for the word associations subtest. Having to recall, under time pressure, the maximum number of nouns referring to animals or means of transport is not a task which assesses basic language skills. As recalling large quantities of semantically-related words under pressure differs greatly from the way words are retrieved in the natural process of sentence formation, the task involves its own cognitive strategies. Once again, the lack of theoretical rigour in implementing what the test set out to do is observed and evidenced by an additional passage of the technical manual: “(D) although it seems a relatively “pure” measure of content, the word associations subtest also provides some insight into strategies employed in the recall of words stored in long-term memory, including planning and grouping strategies” (CELF Technical Manual, p. 24). Therefore, by the authors’ own description, the inclusion of such tasks on a test which allegedly assesses basic language skills brings into question its validity and effectiveness for identifying deficits in these basic skills.
2.5.4 The Peabody
The Peabody (Dunn, 1965) is a test of vocabulary assessment with a picture selection format. It is, together with its British equivalent (the British Picture Vocabulary Scale, BPVS), widely used in the English speaking world, and has been translated into many languages. In general, the criticisms that have been made about the TOLD’s subtest ‘picture vocabulary’ are applicable to the Peabody. Corrêa, Freitas and Lima (2003) observed that the Peabody does not offer a systematic list of stimuli, in particular, grammatical category was not taken into account when selecting the items for the task. There are uneven numbers of nouns and verbs, so the test does not allow the therapist or the researcher to evaluate the types of lexical items that the child has acquired. In addition, like the TOLD’s ‘picture vocabulary’, many of the test items in the Peabody are highly dependent on schooling and access to formal, written language or specific topics, for example ‘lethargic’, ‘ornament’, ‘lobe’, ‘sepal’, ‘mendicant’, ‘edifice’, ‘quiescence’, ‘walrus’, ‘jurisprudence’, ‘indigent’ and many others. Moreover, many of the test items are very difficult to represent pictorially, for example ‘convergence’, ‘astonishment’, and ‘constrain’. In these cases, the mapping of the lexical meaning with the picture can potentially be the main hurdle to overcome in completing the task. Corrêa, Freitas & Lima (2003) argue — and I endorse their
Խ
슀
view — that the Peabody does not provide an informative measure of language development.