5.7 Conclusion
The chapter has presented STIR, a state-of-the-art incremental repair detector that can be used to experiment with incremental performance and accuracy trade-offs. I have shown its efficacy on the Switchboard test data. Its primary features are information-theoretic ones which are accessed in a psycholinguistically plausible time-linear process. The word-by-word incrementality and efficiency for which it was designed allows it to operate with no latency and high incremental accuracy, which will be a useful feature for interactive dialogue systems.
STIR also shows some promise in becoming a domain-general incremental repair detec- tor and therefore of practical use, in this case being used in psychiatric consultation transcripts though it is also currently being employed for other repair annotation tasks. In future work in addition to incorporating acoustic data, I plan to include probabilistic and distributional features from a top-down incremental parser e.g. (Roark et al., 2009), and use STIR’s distributional features to classify repair type to make it more robust and explore the information-theoretic paradigm for incremental processing further.
Chapter 6
An Incremental Semantics Driven Model
of Self-Repair Processing
In this chapter I describe an abstract formal model for incremental self-repair processing as it functions within Natural Language Understanding (NLU) and Natural Language Generation (NLG) in a dialogue model. I also describe methods for implementing this model into the di- alogue systemDyLan(Purver et al., 2011, Section 3.3). I argue that the framework and imple- mentational methods laid out here address lacunae in previous formal computational models of self-repair described in Chapter 3.1
6.1 Self-repair requirements for an incremental dialogue model
For a dialogue model or system to have incremental self-repair processing capability, given the empirical evidence hitherto presented, it should be able to parse and generate repaired utterances like the following at no greater processing cost, and with the same degree of strong incremental interpretation and grain of incremental representation as for fluent utterances, without filtering the effects on fluency (the edit terms and reparanda) out of the input:
(6.1) “But one of [ the, + the ] two things that I’m really. . .”
Repeat (sw4356)
(6.2) “John goes to Paris,{ uh,} from London”
6.1. Self-repair requirements for an incremental dialogue model 168
Extension (or forward-looking disfluency) as an editing term (constructed example)
(6.3) “. . . but my kids are only elementary [ grades, + levels ] right now” Substitution (sw4325)
(6.4) “. . . the bank was suing them [ for, +{ uh,} ] because they went to get. . .”
Delete (sw4356)
(6.5) “[the interview was, + {. . . } it was] alright”
Substitution with continued access to the reparandum (Clark, 1996, p.266)
(6.6) “Peter went[ swimming with Susan, + {or rather,} surfing] yesterday”
Substitution requiring ellipsis resolution using the reparandum (constructed example from anonymous SemDial 2012 reviewer)
6.1.1 NLU requirements
Several desiderata for self-repair processing in NLU can be inferred from these examples, given the demands of incrementality argued for in the previous chapters.
The first requirement is that self-repair processing should be strongly incremental. Repairs should be detected and assigned a suitable representation immediately upon the repair onset with minimal latency, as the STIR system described in the previous chapter was capable of on a structural level.
Secondly, and perhaps the core claim of this thesis, NLU should be able to incrementally interpret the type of repair made in terms of its contribution to the meaning of the utterance in the dialogue situation; this information should be made available with strongly incremental in- terpretation (maximal semantic information) within an appropriate framework, and not in any way be filtered out. While it should be capable of dealing with the surface forms of edit terms, repetitions, substitutions and deletes, or more complex subtypes of these as described in Chap- ter 4, these should processed in terms of their meaning and dialogue function, rather than surface form– this chapter therefore takes a semantics-first approach. For example, NLU must incremen- tally be able to interpret isolated edit terms as indicating forward-looking trouble for the speaker, and other repairs as backward-looking trouble sources for particular parts of the utterance, in line with Ginzburg et al. (2014)’s proposal for differentiating the interactive meaning of the two types.
e.g. in (6.5),“the interview” needs to remain accessible when parsing “it” for anaphora resolu- tion to function, so it must also keep track of processing context and re-use parts of the semantic and syntactic context built up by utterances appropriately. This is particularly the case in repairs containing ellipsis like (6.6), where, given a suitable context, “with Susan” should be incorpo- rated into the asserted information rather than discarded and can be seen as an element present implicitly when processing the repair phase “surfing”.
Fourthly, given the possible parsing ambiguity in this last example, an NLU model must tightly interact with dialogue context, which may be available from other sources of discourse information in the dialogue framework, in order to select the most likely interpretation. So the repair interpretation process, whilst clearly having a close interaction with syntactic processing, should function in an interleaved fashion by querying the ‘higher-level’ conceptualisation part of the framework. In the same vein, for NLU to interact with NLG, it must also be able to stop parsing at any given point and have an accessible context ready to be used by NLG. While this interchangeability is more of a general constraint for an interactive framework (see Purver et al., 2014), its specific role for modelling self-repair will be explained below.
Finally, the NLU repair mechanisms should incur no greater processing costs than fluent utterances wherever possible. This is not only a practical stipulation for their implementation, but is also consistent with psycholinguistic evidence (Brennan and Schober, 2001, Section 2.4.3).
6.1.2 NLG requirements
The desiderata for an NLG account share the five just described of NLU (but in a generative ca- pacity for the mentioned phenomena), with the additional requirement that its interaction with a conceptualiser may include changing generation inputs at any given point during the interaction, and must deal with such changes with appropriate processing and output behaviour. In the spirit of (Guhe and Habel, 2001; Guhe, 2007), communicative goals may be incrementally constructed by a conceptualiser (or dialogue manager) and passed to the tactical generation processes, so an efficient mechanism must be in place to generate required repairs to reflect the conceptual changes in the most natural way possible. As discussed in Section 3.3.1, Skantze and Hjal- marsson (2010) began to address this, but their system, lacking an incremental semantics and the requirements listed for NLU above, lacked the facility for representing the discourse effects caused by generating self-repairs, nor could their system construct representations that could be worked on by NLU. Buß and Schlangen (2011)’s model represented repair events, though
6.2. Background: DS-TTR, the IU framework and DyLan 170