Another point to discuss concerns the compression rates and the relation between compression rate and F-measure. The compressions generated with RASP are considerably shorter than
8.8 Summary 141
those generated with the Stanford parser. This is mainly due to the fact that the structure output by RASP is not necessarily a tree or a connected graph. In such cases only the first subtree of the sentence is taken as input and compressed.
The worse performance of RASP-based system in comparison with the Stanford parser- based one can be accounted for by the lower accuracy of RASP, as we have shown (Sec.8.7.2.1). However, it can also be attributed to a lower compression rate6. Indeed, it seems that F- measures should not be compared on their own but only if they are obtained with identical or nearly identical compression rates. For example, a trivial baseline which does not compress sentences at all has a compression rate of 100% and an F-measure as high as 61.4% (average precision and recall are 53.9 and 74.0, respectively).
8.8
Summary
In this chapter we presented an unsupervised method which compresses dependency trees and not strings of words. We have argued that our formulation has the following advantages: firstly, the approach is unsupervised, the only requirement being that there is a sufficiently large corpus and a dependency parser available. Secondly, it requires neither a subcategoriza- tion lexicon nor hand-crafted rules to decide which arguments are obligatory. Thirdly, it finds a globally optimal compression by taking syntax as well as word importance into account.
Chapter 9
Conclusions
The popularity of text summarization has been steadily increasing in recent years. This is not surprising given its practical utility: multi-document summarization systems would be of great use given the enormous amount of information daily appearing online. In the introduction we demonstrated how multi-document text summarization could benefit from sentence fusion. In particular, we argued that the wide-spread extractive strategy falls short on resolving the non- redundancy/completeness problem and that a system capable of generating new sentences directly from text would be very useful. We gave an overview of existing summarization systems which go beyond extraction and pointed out their shortcomings. Furthermore, we showed that to date there is no implemented method which would fulfill all of the following requirements:
• generate complete, grammatical sentences;
• produce sentences which are not only grammatical but also consistent with the input; • make the length of generated sentences adjustable so that succinct as well as detailed
sentences could be produced, dependent on the application.
This thesis presented a novel sentence fusion system, deFuser, developed for and tested on German data which fulfills the above mentioned requirements. Given a collection of related documents, deFuser identifies and groups similar sentences. It combines relevant information from different sources and produces new grammatical sentences; it can also be used for sen- tence compression. These two properties makes deFuser an attractive tool for multi-document summarization. In this final chapter we summarize the main contributions of our work and outline possible directions for future research.
9.1
Main Contributions
Syntactic and semantic well-formedness. deFuser generates novel sentences by, first, build- ing a grammatical and semantically sound syntactic structure, and second, by finding the best word order for this structure. With regard to these two main steps, it is important to emphasize the following contributions:
1. Grammaticality of the dependency tree is ensured with a few language-independent constraints – neither hand-crafted rules, nor resources unavailable for most languages are required. Relative grammatical importance is determined from a parsed corpus and is used not only for verb arguments but for all words and dependency relations.
2. Apart from grammaticality, which concerns the syntactic side of sentence well-formedness only, deFuser implements a semantic component and checks semantic compatibility of co-arguments. Such a semantic module is missing in most text-to-text generation archi- tectures which use semantic information for similarity identification only.
3. We introduced a novel tree linearization method with considerably less overgeneration than previous approaches. On German data our method outperformed several baselines. We argued that a combined method which separately finds the best order of constituents on the clause level and the best word order within constituents on the phrase level is beneficial. Our experiments confirmed that trigram language models are appropriate for phrase linearization but are not useful on the clause level where long-distance depen- dencies are typical and more linguistic information is required. We showed that our combined method works equally well on English as well as on German data.
More abstractive fusion. In contrast to previous sentence fusion systems, deFuser abstracts from single sentences to a global representation – a graph of dependencies. This graph is built from transformed syntactic dependencies and reveals semantic relations. It is this semantically and syntactically motivated representation from which deFuser generates new sentences. This is important, because it brings summarization one step closer to abstraction. As a consequence, generation of sentences which incorporate information from different sources becomes pos- sible. Abstracting from sole dependency trees to one global graph also makes the generation task more challenging because a single correct tree is no longer available in the graph.
Applicability to Sentence Compression. We demonstrated that deFuser can also be applied to sentence compression. Tested on English and German data, it showed performance compa- rable to recently developed systems. Importantly, unlike the majority of sentence compression methods, deFuser is unsupervised and requires an accurate dependency parser only.