Coherence models approach discourse as a collection of discourse segments connected by underlying coherence relations. Coherence analysts have attempted to categorize and codify these relations (see §2.1.2), but their approach to this endeavor is largely intuitive. Mann et al. (1989: 19) take up this point explicitly and admit that the type of analysis performed in RST is “necessarily subjective”, but that no other approach is available. They concede, to their credit, that there are “methodological risks” associated with their approach:
“risks of circularity, divergence of analysis from actual function, nonrestrictiveness of the theory, vagueness, indefiniteness of analytic outcome, etc.” (ibid.)
The main problem with their approach, however, lies in the subjective judgment of which coherence relation fits in a given context. As Mann et al. put it (id.: 32),
“It often happens that a text has more than one analysis; it is a normal and predictable outcome, given the way that RST is defined. We and others have had the experience of giving the same text to several analysts, who then created differing analyses, sometimes more than one from an individual analyst.”
This is, of course, a pretty significant obstacle. If the tools available under a certain framework do not (or cannot) lead to, at the very least, a consensus on which coherence relation applies in a given context, the coherence program’s practical applicability seems inherently limited. Spooren & Degand (2010: 242) expand on this problem of potentially low interrater (or intercoder) agreement:
“This type of analysis requires a large number of coding decisions, which are usually based on subtle interpretive differences. Hence they raise the issue of intercoder agreement. […] Low interrater agreements suggest that the categories used in a theory are vague, in the sense that categorizations are non-replicable, and consequently unfit as a basis for theory building. Despite its importance, there is presently no tradition in the field of corpus-based discourse studies to report agreement measures.”
They continue by pointing out that coherence approaches often do not provide “explicit accounts of interrater agreement”, and that we are hence left in the dark as to how reliable a certain interpretation is (id.: 243). If a discourse segment is taken to indicate Evidence, did all analysts agree? Or did others see it as a Background, or an Elaboration? How many analysts agreed on a given relation? How many relations were proposed for a given utterance? How many analysts were there in the first place? These are important questions, which are usually left unanswered. Spooren & Degand (id.: 254) also point out that where interrater scores are provided, “high agreement scores are rare in the case of the analysis of coherence relations.”1
Spooren & Degand (2010: 254-255) suggest that part of the reason for low interrater agreement when parsing texts for coherence relations is the influence of context. In order to establish the correct coherence relation, they argue, the analyst (or the hearer) has to supply “contextual information, which in itself can be interpreted in multiple ways”. Take the following example (from Lascarides & Asher 1993: 438):
(1) The council built the bridge. The architect drew up the plans.
Lascarides & Asher (1993: 440) take this to be an instance of ‘Elaboration’, where the second clause (i.e., discourse segment) provides information about the ‘preparatory phase’ preceding the building of the bridge in the first discourse segment. However, one could also parse the relation as being one of Contrast, given the right context. For example, let’s say a councilman is arguing with someone about who is responsible for the bridge. His interlocutor believes that a large part of the credit should be due to the architect, while the councilman, of course, believes that the council should be awarded the lion’s share of the plaudits. In this context, the councilman would utter (1) to make a contrast between the council, who built (i.e., paid for) the bridge, and the architect, who (only) drew up the plans. As such, there is ambiguity between coherence relations here, which can only be resolved if the right (i.e., speaker-intended) context (and other potentially available information such as prosody) is brought to bear on the interpretation of the utterance.
In (1), then, the analyst needs the right contextual information so that he can establish the right coherence relation, so that he can (finally) establish the correct interpretation of the text (see also Blass 1993: 92). As we shall see, however, it is not at all clear why we need an extra step – if the hearer is able to supply the right (i.e., speaker-intended) contextual information, doesn’t this suffice to establish the right (i.e., speaker-intended) interpretation of the text? Take example (1) again: if the hearer is
aware of the context I outlined under the example, does he really need to determine that there is a relation of Contrast between the two discourse segments before he can understand that the speaker is implying that the council deserves more credit for the construction of the bridge? Isn’t it possible that, by combining the contextual information provided under the utterance in (1), the hearer is able to understand the speaker of (1) to mean that the council should take a large part of the credit for building the bridge, and that their financial support of the bridge trumps (i.e., should be contrasted with) the planning performed by the architect? There are strong reasons to assume that this is possible, and that we can ‘cut out the middleman’ – in other words, that we do not need to establish the right coherence relation in order to establish the right interpretation, but that we can proceed from establishing the speaker-intended context to establishing the speaker-intended interpretation without coherence mediation (see e.g. Blass (1990) and Blakemore (2001a), to be discussed shortly). Given that this would presumably be a more cognitively efficient way of processing discourse (and provided, of course, that this alternative account is plausible), there would have to be robust arguments in favor of the position that the recognition of coherence relations is an essential component of utterance interpretation. In light of the low interrater agreement discussed here and the claims against coherence to follow, it is not obvious how such an account would go.2