The present series of ERP experiments on auditory sentence processing yielded new insights into the development of children's sentence processing mechanisms. Moreover, the data also had consequences for models on language processing in adults. Specifically, the children's ERP data as well as differences between audi- tory and visual sentence processing led to new conclusions about the characteris- tics of adults' sentence comprehension.
The children's ERP data had a most important influence on the interpretation of the ERP components in adults. This primarily concerned the N400 component. Originally, the N400 effect in ambiguous dative-initial sentences was interpreted as a reflection of the relabeling of base generated syntactic positions (Bornkessel et al., 2004). It was assumed that the existence of canonical dative-initial struc- tures in object-experiencer verbs was somehow overgeneralized in order to re- analyze object-initial sentences with dative active verbs, too.
Crucially, the data of the 11-year-old children (Experiment II) were not easily reconciled with the assumption of overgeneralization. At 11 years of age, only active verbs elicited an N400 effect, but object-experiencer verbs did not. Instead, object-experiencer verbs showed a P600 effect, compared to subject- initial active conditions. This indicated that the N400 seen in adults cannot origi- nate from object-experiencer verbs.
Alternatively, the N400 could be explained within the macrorole account of Van Valin and La Polla (1997). They assume that all dative verbs are intransitive and assign only one macrorole (Actor or Undergoer). Consequently, the N400 effect may reflect the reassignment of this single macrorole to the dispreferred second argument. The subsequent P600 would then reflect the reanalysis towards the non-canonical word order in active verbs. For object-experiencer verbs, no further reanalysis is necessary, as they are associated with a canonical object- initial word order. Here, the macrorole reassignment suffices, which is reflected in the N400 effect.
A second point that had an important impact on the understanding of adults' lan- guage comprehension concerned data on modality dependent differences in sen- tence processing. All experiments in this thesis were conducted in the auditory
modality in order to avoid a confound of differing reading proficiencies between children and adults. Crucially, all previous ERP studies on the role of word order and case marking in German sentence processing were conducted in the visual modality. Thus, the adult control data turned out to be a test case for the gener- alizability of previous visual data.
Table 10.2 Overview of adults' ERP effects on the critical verb, separately for different verb
types; results of the visual domain stem from Bornkessel et al. (2002; 2003; 2004); results of the auditory domain stem from Experiments I and IV, respectively; 'verb type effect' is the ERP effect for dative object-experiencer verbs in comparison to dative active verbs; 'word order effect' is the ERP effect for ambiguous object-initial structures in comparison to ambiguous subject-initial structures.
condition visual modality auditory modality verb type effect: object-experiencer vs. active verbs
case unambiguous EPOS EPOS
case ambiguous
word order effect: object-initial vs. subject-initial sentences
accusative P600 N400 P600
dative active N400 N400 P600 dative obj.-exper. N400 N400
The results from experiments on adults' sentence processing in the visual (Bornkessel et al., 2002; 2003; 2004) and in the auditory modality (Experiments I and IV) are summarized in Table 10.2. As in the visual modality, adults used morphological case markings in order to rank the arguments of a sentence the- matically in the auditory experiment (Experiment I). As in prior visual experi- ments, the reversal of the thematic hierarchy in dative object-experiencer verbs was reflected in an EPOS effect, independently of word order. In auditory pres- entation, the EPOS effect emerged 100 ms later than in visual presentation. Such differences in latency could easily be explained by the fact that auditory stimuli unfold over time. Thus, it took longer to unambiguously recognize an auditorily presented word than a visually presented word. In terms of the ADM, adults use the Morphological Pathway to establish thematic dependencies if the arguments are unambiguously case marked.
If the arguments are ambiguous with regard to case, adults use word order to establish a syntactic hierarchy of the arguments. In terms of the ADM, the
processing of ambiguous arguments takes place on the Positional Pathway. The disambiguation towards an object-initial word order required reanalysis.
Crucially, the ERP patterns elicited by reanalysis differed between modalities. This was the case for dative and for accusative structures. Whereas the reanalysis of dative structures elicited only an N400 effect in the visual modality, dative active verbs additionally elicited a P600 effect in the auditory modality (Experi- ment I). With accusative verbs, all visual studies reported a P600 effect for the reanalysis of ambiguous object-initial sentences of the type examined here (e.g. Bornkessel et al., 2004; Friederici et al., 2001). By contrast, in auditory presenta- tion surprisingly a clear N400 effect emerged that was followed by a P600 (Ex- periment IV).
The ERP differences in dative structures were most likely due to the time point at which specific types of information become available. Whereas in visual presentation the whole verb was displayed at once on the screen, in auditory presentation the verb unfold by and by, making verb type information available before agreement information (e.g. 'zuwink-t' vs. 'zuwink-en'; wave to, singular vs. plural). Hence, verb type specific information had time 'to sink in', before the sentence structure was disambiguated. In consequence, verb specific information could be activated before reanalysis took place. In the case of object-experiencer verbs, verb specific information supported reanalysis, as the dative-initial word order is the canonical one. By contrast, no such support was given in dative active verbs as they require a canonical subject-initial structure. Hence, reanaly- sis towards an object-initial word order was not supported with active verbs and thus elicited the additional P600 effect. The same ERP pattern also emerged in a visual experiment, in which the information types were pulled apart by present- ing the verb before the disambiguating arguments (cf. Bornkessel, 2002).
However, such 'physical' explanations cannot account for the modality differ- ences in accusative verbs. In contrast to the P600 reported in visual experiments, ambiguous accusative-initial structures elicited an N400-P600 pattern when pre- sented auditorily. An N400 effect in connection with syntactic reanalysis had only been reported for the reanalysis of dative structures. The reanalysis-N400 in dative verbs may, for instance, reflect the reassignment of a single macrorole, as dative verbs are intransitive. Nevertheless, this type of reanalysis is not possible with accusative verbs, because they are associated with two macroroles (Van Valin & La Polla, 1997). Crucially, the reassignment of two macroroles is a
fundamentally different process, which in visual presentation was thought to be reflected in the P600 effect.
The observed differences in the processing of accusative structures are diffi- cult to account for under a 'modular' interpretation of the ERP components N400 and P600. Typically, each ERP component is associated with a specific type of cognitive process (cf. also Chapter 2). Within such a modular account, the ap- pearance of a new component would indicate a new underlying process. In the present case, the emergence of the additional N400 effect in the auditory modal- ity would be interpreted as reflecting an additional cognitive process that was not present in all preceding visual experiments. Interestingly, recent fMRI studies also indicated that the neural network engaged in the processing of sentences with word order permutations differs depending on the input modality (cf. Fie- bach et al., 2004; Röder et al., 2002).
However, the question remains why participants should apply different sen- tence processing mechanisms when reading compared to when listening to a sentence. It seems as if a modular account does not help in understanding the dif- ferences and commonalities of the ERP patterns in auditory and visual sentence processing.
The ERP pattern shall be summarized once more. It is always the case that re- analysis with dative verbs elicits an N400 effect, and reanalysis with accusative verbs elicits a P600 effect. In addition, the ERP effects may overlap, which re- sults in an N400-P600 pattern. This was the case with dative active and with accusative verbs in the auditory modality. Perhaps one would need another level of abstraction in order to describe the specific function of each component and to explain why they may overlap under certain circumstances. Moreover, the condi- tions under which such overlaps occur need to be investigated in further detail. For instance, overlapping ERP patterns might not only occur in auditory presen- tation, but rather depend on a yet unknown third factor.
The investigation of auditory language comprehension is essential to develop- mental research. Children enter the language system by listening to their native language from the first day they are born. They start to talk long before they learn how to write. Thus, in order to track language development from the very beginning to the highly sophisticated adult processing mechanisms, research is naturally bound to the auditory modality. However, work on adult sentence proc-
essing predominantly focused on the processing of written sentences. Visual studies form the basis for many sentence processing models, often neglecting the auditory modality. As such, they provide an incomplete basis for research on language acquisition. The results of this work indicate that visual data do not simply reflect amodal sentence processing principles that can directly be trans- ferred to the auditory modality. Rather, auditory sentence processing needs to move to the focus of language processing research in order to enhance investiga- tions in language development.
In future studies, a systematic comparison of modality differences in sentence processing will be necessary. As discussed above, data on auditory sentence processing are underrepresented, a fact that renders research on language acqui- sition quite difficult. Thus, it would be fruitful to illuminate differences between auditory and visual modality. For instance, the role of prosodic information in auditory sentence processing should be investigated in greater detail. Acoustic analyses of the stimulus material indicated that word order differences are partly reflected in the duration of a sentence's phrases. On the other hand, verb type in- fluenced the course of the fundamental frequency. It might be the case that such prosodic cues affect the parser's decisions right from the outset of a sentence (cf. also Eckstein & Friederici, submitted). Balanced experimental designs could dis- entangle the influence of the input's physical properties and profound differences in processing routines between auditory and visual modality.