SUBTOTAL MOBILIARIO

Several approaches which apply the idea of distant supervision to solve NLP tasks other than relation extraction have been proposed, which demonstrate the usefulness of distant supervision beyond relation extraction. Note that the term “distant supervision” is used very loosely for some of those applications.

Marchetti-Bowick and Chambers (2012) identify keywords related to political subtopics (e.g. Obama, Ideology) and make the assumption that if that keyword occurs in that tweet, it is about that topic. In addition, they use sentiment words to identify the sentiment of each tweet. The combination of the two, topic keyword and sentiment word, leads to an aspect-based sentiment analysis approach. Unlike distantly supervised relation extraction aproaches, they do not use tweets containing both words for training one classifier, but first train a topic classifier, then train a second classifier for sentiment on topic-relevant tweets. For both stages, a multinomial Naive Bayes classifier is used. For topic identification, findings are that for a Twitter corpus only containing political tweets, a very high F1 score of around 90% can be achieved with such an approach, but for general tweets, only an F1 score of around 18% can be achieved, mostly due to precision being around 10%. At the second stage, sentiment classification, it is shown that the aspect-based sentiment analysis appraoch with distant supervision outperforms a lexicon-based approach. While the reported results show the benefit of the distant supervision idea, the authors do not discuss why they opted for a two-stage classification approach instead of labelling tweets with both arguments, then training a classifier, as for distantly supervised relation extraction. It would be interesting to see how this would compare to the two-stage setting.

Exner et al.(2015) propose to use distant supervision creating semantic role labelling resources in languages other than English, then training a semantic role labeler on those resources. They start with the English version of PropBank and English Wikipedia. The goal is then to identify

propositions in the Swedish Wikipedia, e.g. they try to translate the English predicate “win.01” to Swedish predicate “vinna.01”. Using Wikipedia disambiguation pages and external named entity linking tools, they map mentions of NEs to unique Wikipedia-based identifiers in both English and Swedish Wikipedia. They then identify propositions in the English version of Wikipedia and for sentences which contain them, get the unique Wikipedia-based identifiers. The distant supervision idea of the approach is to then use those pairs of named entities linked to identifiers with a proposition to identify pairs in the Swedish Wikipedia. Because they use the same Wikipedia- based identifiers for NEs via cross-language links, the propositions can be transferred from one corpus onto the other corpus in a straight-foward way, by identifying sentences which contain the same pairs of identifiers. The sentences with automatically aligned propositions are then used to train a semantic role labeler. The overall idea of the approach is very similar to distant supervision for relation extraction, with the difference that the corpus, in that case Swedish Wikipedia, is not directly annotated with the background knowledge base, but indirectly via a semi-parallel corpus, English Wikipedia. This makes the task more challenging than distantly supervised relation extraction. However, they still report reasonably high results – a precision of 58% at a recall of 47%.

Parikh et al. (2015) experiment with using the distant supervision idea to train a semantic parser. Whereas relation extraction focuses on extracting binary relations, semantic parsers learn to recognise additional semantic relations such as “cause” or “theme”. Thus this can be seen as a more complex form of knowledge extraction, where the event structure has the representation of a semantic parse. In the case of events, it is very difficult to find all arguments of events in one sentence. Instead, they decompose the events into subevents, then later augment the local events. They evaluate on the GENIA event extraction shared task data (Kim et al., 2009), on which they outperform 19 of 24 submissions. The distantly supervised approach alone achieves a precision of 29.4% at recall of 19.1%. What brings big improvements is collecting five trigger words for each event and incorporating them into learning, which radically improves results to a precision of 72.2% at a recall of 27.9%. It is interesting to see that trigger words bring such a big improvement for distantly supervised event extraction. Maybe this is something that could also bring improvements to distantly supervised binary relation extraction, i.e. to manually define trigger words for relations.

Magdy et al. (2015) use the idea of distant supervision and apply it to classifying tweets into topics with the help of YouTube labels. They do so by collecting tweets which contain links to YouTube videos and then retrieving the topic the video is assigned, which is one of 18 coarse- grained classes. They merge those to 14 classes, thereby avoiding too sparse or too general classes, and end up with categories such as “Pets & Animals”. A topic classifier is then trained on such tweets containing YouTube links and can be applied to other tweets which do not have to contain YouTube links. In practice they perform a hold-out experiment. The overall idea of using distant supervision is similar to that of Marchetti-Bowick and Chambers (2012), apart from that they do not train a sentiment classifier afterwards. They do not test their approach on general tweets as Marchetti-Bowick and Chambers (2012) do, instead only tweets which they already know to contain one of the topics. As such, their results are relatively high, around 57% precision and