A.2
Story-driven event annotation typologies
The following are dynamic typologies constructed for event domain (Figure A.2) and type (Figure A.1) for and during the story-driven event annotation task (Section 3.3). Note that these are produced organically and would in practice be refined after the annotation as we have done for named entity types (Nothman et al., 2013).
Entertainment Music Literature Film Classical Contemporary Federal Crime Environment State Private International Local Public Economy Retail Interest rates Employment Property Business/Finance Government Migration W orld Human interest Politics
Science & technology
Sport Disaster Civic Preservation Health Education Domain
Figure A.1: Hierarchy of event domains. Bold arcs were added during the annotation; others were initially provided.
Ceremony Accident T ournament End (Death/Decommissioning) Contest Justice Arrest (/Char ge) Hearing
Submission (to a case)
Investigate V erdict (/Sentence) Legal Sponsorship Regulation T ransaction Contracts A ward Loan Payment Marriage Alliance (/T reaty/Agreement) Employment Purchase Innovation Injury Conflict Protest Critique Physical Discovery Measured change Gain Loss T rend Begin (Birth/Foundation/Creation) Inspect Mer ger Compensation Settlement
(of legal case)
Inauguration Nomination Unemployment Bankruptcy (/Liquidation) Construction Lobby Product release Lifecycle Interaction Physical assistance Negotiation Scandal Disaster Show R&D Invention Unstructured event Structured event Election Event
Figure A.2: Hierarchy of event types. Bold arcs were added during the annotation; dotted arcs were deleted from the initial hierarchy; others remained throughout the annotation.
Appendix B
Annotation schemas for event
linking
This appendix reproduces the annotation schemas for our pilot (Section B.1) and final (Sec- tion B.2) event linking annotations, as well as the worked example (Section B.3) employed for annotator training (refer to Chapter 5).
We note some differences in category labels between the schema and the body of the thesis in Table B.1.
In schema In body
Basic event linkable
Complex event compound
Trend or measured change aggregate
Many / multiple / generic multiple
Table B.1: Equivalence between labels used in the schema and the main work
B.1
Pilot annotation schema for event linking
Task: link each expression which refers to a newsworthy event that has happened (from the perspective of the expression’s author) – and which the informed reader would know of from previous news – to an article representing that event.
The idea is to decode language which refers to events as a pointer to something in the reader’s knowledge/memory of events, assuming that the reader has read and remembered any event in the archive that someone might later refer to.
What to mark for linking
Ideally, one is to mark all expressions that refer to newsworthy events the reader is likely to have known about (i.e. from previous reporting of that event). But defining what constitutes an event for this purpose, both textually and semantically, is difficult.
Events that are first reported as news in the article being annotated should not be marked. (Even though they are referring expressions, we will concern ourselves only with the commu- nicative pragmatics of referring to a previously known event.)
Extent
We annotate expressions which denote a (newsworthy) event that happened. We are partic- ularly interested in expressions the author uses to refer to events that a regular reader would know about from prior news.
Generally, annotation spans will be:
• a single head verb or noun
• which predicates the event / bears the event-denoting content (not merely tense, etc.) • for rigid designators and proper nouns, annotate only the head
Auxiliaries (e.g. has/had in X attacked) should generally not be marked. The following should not be marked:
Implicit references
Quasi-events Expressions referring to change in some measured value (e.g. prices rose) Introduced here Expressions denoting events which are being reported first in the current
article (i.e. they are not being used to refer back to existing knowledge). If at first you aren’t sure if an event was introduced in the present article, mark it, and if you conclude that it was, indicate such under “why no link?”.
The following should be marked, but not linked (see below):
Plural Expressions which refer to many event instances that would have been reported
individually, such as the recent wave of attacks. . . attacked on Monday and again on Tuesday, or his spate of injuries. While these may refer to a specific array of past events, we have no way to encapsulate them in our knowledge base. (Nonetheless, Sunday’s demonstrations may be linked despite events occurring over a multitude of locations.) This also includes another in residents fear another attack (implies a reference to a past event in addition to hypothetical events), and complex events which would be difficult to pinpoint to a particular article, such as an election or sports season.
Semantic class
A relatively precise discussion of event-referring language is given in the TimeML event annotation guidelines (Saur´ı et al., 2009) which divides all verbs and other event predicates into:
REPORTING say, according to . . . PERCEPTION see, hear . . .
ASPECTUAL begin, complete, (dis)continue . . . I-ACTION try, cancel, investigate, order, nominate . . . I-STATE feel, desire, fear, require . . .
B.1. Pilot annotation schema for event linking 177
STATE have . . .
OCCURRENCE arrive, explode, distribute . . .
Generally we are interested in events that TimeML would class as Occurrence and As- pectual. So for our purposes, generally:
• someone saying something is a non-event
• someone hearing or seeing something is a non-event • someone having something is a non-event
• someone feeling or wanting something is a non-event
• someone proposing something we still haven’t decided on. . .
• someone continuing to do something is a non-event, despite being driven by an aspectual
event
There may be exceptional cases, where one of these activities was in itself a newsworthy event, in which case it should be marked, and a note made to that effect in the comments.
Nonetheless, we are less concerned with these semantics than with the pragmatics of how the expression is used to refer to a known and newsworthy past event.
Coreferring expressions
Mark all expressions in separate sentences which refer to the same event as coreferent. How- ever, unlike in other event annotations, coreference does not need to be precise, as long as the events have the same link.
What to link to
Treating the archive as a knowledge base
We are only interested in linking to stories that introduce (to the reader’s knowledge) one or more events. As such, link targets should always be news articles designed to report events (and the event reported should exist outside of the article).
We only consider the first article which refers to the event having happened. Articles which posit the event’s future occurrence are not canonical representatives of that event in the knowledge base.
Not found in the archive (kb)
An event reference may not refer to an article in our knowledge base, either because it precedes our archive, or no appropriate article exists.
Mark such expressions, and instead of linking, select the appropriate response from the “why no link?” drop-down.
“Why no link?”
There are some cases when expressions should be marked but not linked, some of which have been discussed above. We provide a widget to indicate why no link is given:
Plural reference the expression can’t be pinpointed to a single reporting
Not found the event should probably be in the archive, given its time and topical scope,
but was not found.
Precedes the archive the smh archive only goes back to 1986, so nothing can be linked
before then
Introduced here indicates that you had thought the expression might refer back to past
news, but in fact you found it was first reported in the present article.