hypoactive NNP suburbs said specification today
plasticky woman integral was auditory system
psychogenic problems collective do pal programs
yoplait television physiologically do steganography programs
subminimal level amuck run wastewater system
ehatchery investment illegitimately put autism conference multistage process straighter make timbre changes aquacultural products secret talk pulmonology department
caplike form holy keep monkeypox case
apomorphine treatment cerebrally felt monkeypox cases antispam operations norepinephrine knew strontium levels
SUBJ-VERB
blog said briefer said hr said
knucklehead said lymphedema have permissions have
steganography have monkeypox had ipso is
neuroscientist said cybertrainer make
Table6.3: Unusual word-pairs from different categories
Based on our observations on the development set, we picked a cutoff of0.0001on the probability (0.001for adverb-verb pairs) and consider phrases with probability below this value as unusual. For each test article, we compute the number of unusual phrases (total for all categories) as a feature (surp) and also this value normalized by total number of word tokens in the article (surp wd) and normalized by number of phrases (surp ph). We also compute features for individual pair types and in each case, the number of unusual phrases is normalized by the total words in the article (surp adj noun, surp adv verb,
surp noun noun,surp subj verb).
A list of the top unusual words under the different pair types are shown in Table
6.3. These lists were computed on pairs from a random set of articles from our corpus. Several of the top pairs involve hyphenated words which are unusual by themselves, so we only show in the table the top words without hyphens.
All these features are different between the two categories as expected.
Higher invery good: avr phoneme perp all,avr char perp(all,10),surp,surp ph,
surp wd,surp adj noun,surp noun noun,surp subj verb
Higher in typical: freq nyt
The average perplexity of words from thevery goodarticles is higher under both the character and the phoneme models. The average frequency of these words in the back- ground corpus is also lower. For the word pair based features, the proportion of unusual phrases is also higher in the very good articles. These findings indicate that unusual word phrases as hypothesized are associated with the good samples in our corpus. 6.1.4 Sub-genre
This aspect differentiates articles at the organization level and abstracts away from indi- vidual words and sentences. There are several sub-genres in science writing [152]: short descriptions of discoveries, longer explanatory articles, narratives, stories about scien- tists, reports on meetings, review articles and blog posts. We expected that some of these sub-genres could be more appealing to readers. For example, a narrative may be more interesting to a reader as he can involve himself with the story line and characters. A snippet from a narrative article in our science journalism corpus is shown below.
Mr. Jousse became one of the world’s foremost urban lighting experts by accident. A native of Paris, he landed a job in1963 with the city’s engineering division after graduating from college, helping widen and deepen the city’s canals. He later had jobs supervising3,000 garbage collectors and creating pedestrian streets. In1981, a supervisor asked him to change course once again.
There are several studies on genre prediction amd mostly on the news domain [72,
121]. These methods use part of speech, pronouns and stop words as features. Rather than include features that are related to genre differences we choose to directly compute scores for some genres of interest in our corpus. We compute simple measures to indicate three genres—narrative, attribution and interview.
Narrative texts typically have characters and events [111]. Based on this idea, we compute a score for the narrative nature of a text based on two factors—entities (pronouns
and proper names) and past tense. We count the number of sentences where the first verb in surface order is in the past tense. Then among these sentences, we pick those which have either a personal pronoun or a proper noun before the target verb (again in surface order). The proportion of such sentences in the text is taken as the score (named
narrative).
We also developed a measure to identify the degree to which the article’s content is attributed to external sources compared to the author’s own statements. Attribution to other sources is frequent in the news domain since many comments and opinions are not the views of the journalist. As we already discussed, theresponsibilityframe which is related to attribution is rather common in news reporting [145]. For science journalism, attribution becomes even more important since the research findings were obtained by scientists and reported in a secondhand manner by the journalists. So we compute a score (attrib) to indicate the level to which the author talks directly about the subject compared to using attributive statements from the scientists. This score is the proportion of sentences in the article that have a quotation mark, or the words ‘said’ and ‘says’.
We also compute a score to indicate if the article is the account of an interview. There are easy clues in NYT for this genre with paragraphs in the interview portion of the article beginning with either ’Q.’ (question) or ’A.’ (answer). We count the total number of ’Q.’ and ’A.’ prefixes combined and divide the value by the total number of sentences (interview). When either the number of ’Q.’ tags is zero or ’A.’ tags is zero, the score is set to zero.
All three scores are significantly higher for thetypicalclass. 6.1.5 Affective content
The writing in an article can also evoke emotions and sentiment in a reader. For example, articles detailing research on health, crime, ethics and well-being can involve and discuss issues that have a lot of sentiment value and be more appealing to a reader. A snippet with high sentiment value is shown below.
”Although it could be argued that there is little to lose in this tragic situation,” he wrote, ”my personal view is that there is a significant risk of causing pain or dis-
tress if the treatment is given and very little prospect of any benefit.” Medicine is a constant trade-off, a struggle to cure the disease without killing the patient first. Chemotherapy, for example, involves purposely poisoning someone – but with the expectation that the short-term injury will be outweighed by the eventual benefits.
We compute features for sentiment value using three lexicons. Two of these, MPQA [172] and General Inquirer [153] give lists of positive and negative sentiment words. The third resource is a set of words associated with emotions and were obtained from FrameNet (Emotion frame) [5]. The sizes of these lexicon are8221, 5395, and 653words respectively. We compute the counts of positive, negative, polar, and emotion words, each normalized by the total number of content words in the article (pos prop,neg prop,po-
lar prop,emot prop). We also include the proportion of emotion and polar words taken together (polar emot prop) and the ratio between count of positive and negative words (pos by neg) as features.
The significant features are listed below:
Higher in very good: neg prop,polar prop,emot polar prop
Higher in typical: pos by neg,emot prop
very good articles do turn out to have more sentiment words. It should also be noticed that the proportion of positive words does not vary between categories but the
very goodarticles have higher proportions of negative sentiment words. A similar trend is thetypical articles having higher values for positive to negative word ratio. However, emotion words are more frequent in thetypicalarticles.
6.1.6 Amount of research content
Science news cannot convey the full depth of research done on a topic in the way that academic publications do. For a lay audience, a science writer chooses the most relevant findings and methods of the research to include in the article and also interleaves the research information with details about the relevance of the finding, people involved in the research and general information about the topic. So the degree of explicit research descriptions in the articles varies considerably.
The study is being published in the April issue of the journal Psychological Science. The findings seem to fall in line with the idea that dreams express complicated de- sires and unfulfilled wishes, as Freud, who called dreams the “royal road to the unconscious,” noted long ago. But Dr. Wegner does not completely agree with that assertion.
To test how this aspect is related to quality, we count references to research methods and research people in the article. We use the research dictionary that we introduced during corpus creation (Chapter3) as the source of research-related words. We count the total number of words in the article that match the dictionary (res total) and also the number of unique matching words (res uniq). We also normalize these counts by the total words in the article and create featuresres total propandres uniq prop.
All four features have significantly higher values in the very good articles which indicate that popular articles are also associated with a great amount of direct research content and explanations.