Synonyms are important as authors and annotators may use equivalent, but differ- ent terminology for the same concept. Synonyms are an essential source to recog- nize ontology terms in text, but also to create a mapping between terms existing in different ontologies. For example, authors might refer to the concept fever in dif- ferent ways. Some texts in Medicine will mention the term “fever” itself, others the Latin name “pyrexia”. The Gene Ontology synonyms “apoptosis” and “programmed cell death” are used synonymously in literature. In Go3R, the term “Bovine Corneal Opacity Test” has synonym “BCOP Assay” with the abbreviated form of “BCOP” fol- lowed by “Assay” as synonym for “Test” in this specific context. Other than in the previous examples, terms are often not exact synonyms, but have slightly broader or narrower senses. Sometimes synonyms are even hypernym or hyponyms. For
Method Characteristics Precision/ Accuracy Comment
patterns machine learning
resources
used Recall Confidence
Landauer and Dumais
(1997) 4
0.64
0.53−0.75
evaluated on TOEFL syn- onyms dataset; using latent semantic analysis
Turney (2001) 4 AltaVista
searches
0.74
0.63−0.83
evaluated on TOEFL syn- onyms dataset, using Point- wise Mutual Information Jarmasz and Szpakowicz
(2003) 4 Roget’s thesaurus WordNet 0.79 0.68−0.87
evaluated on TOEFL syn- onyms dataset
Terra and Clarke (2003) 4 0.81
0.71−0.89
evaluated on TOEFL syn- onyms dataset, using Point- wise Mutual Information Turney et al. (2003) 4 WordNet 0.98
0.91−1.00
joint algorithm of previous 4 methods for TOEFL syn- onyms dataset
Shimizu et al. (2008) 4 0.19/ NA using dependency structure Mccrae and Collier (2008) 4 4
WordNet∗, UMLS∗, Wikipedia∗
0.73 / 0.30
binary classification of synsets with six methods,
∗used for evaluation
Turney (2008) 4 4 0.76
TOEFL synonyms dataset; conjointly finding analo- gies, synonyms, antonyms, associations
Table 2.8. Overview on synonym discovery approaches regarding their characteristics and quality.
The best systems achieve high precision (sometimes accuracy was measured). recall is typically lower, meaning that less then half of all synonyms are found.
example the terms “proof”, “finding”, and “certification” are synonyms for the term “validation”. The term “finding” could be seen as hypernym, “validation” is some sort of “finding”, and “certification” as hyponym, because any “certification” is automati- cally a “validation”.
Table 2.8 summarises nine approaches to find synonyms. These approaches can
Overview on synonym discovery
see Table 2.8
be categorised by their characteristics in using patterns, or machine learning and by the employed external resources. A frequently used benchmark for synonym detection are 80 questions on synonymy of the Test Of English as a Foreign Language (TOEFL). 6 of the 9 reviewed systems compare against this benchmark and achieve a precision between 0.64 and 0.98. All surveyed systems use machine learning to determine synonymy.
WordNet
WordNet is a lexical database of English where nouns, verbs, adjectives and adverbs are grouped into sets of cognitive synonyms (synsets), each expressing a distinct concept (Fellbaum, 1998). For example the noun synsets defined for “learning” (Fig- ure 2.7) distinguish between general learning as cognitive process and the type of learning, like “book learning”. The synsets are interlinked by means of conceptual- semantic and lexical relations creating a network of meaningfully related words and
WordNet entry: “learning” Noun:
• S: (n) learning, acquisition (the cognitive process of acquiring skill or knowledge) “the child’s acquisition of language”
• S: (n) eruditeness, erudition, learnedness, learning, scholarship, encyclopedism, ency- clopaedism (profound scholarly knowledge)
Verb
• S: (v) learn, larn, acquire (gain knowledge or skills) “She learned dancing from her sister”; “I learned Sanskrit’’; “Children acquire language at an amazing rate”
• S: (v) learn, hear, get word, get wind, pick up, find out, get a line, discover, see (get to know or become aware of, usually accidentally) “I learned that she has two grown-up children”; “I see that you have been promoted”
• S: (v) memorize, memorise, con, learn (commit to memory; learn by heart) “Have you memo- rized your lines for the play yet?”
• S: (v) learn, study, read, take (be a student of a certain subject) “She is reading for the bar exam” • S: (v) teach, learn, instruct (impart skills or knowledge to) “I taught them French”; “He instructed
me in building a boat”
• S: (v) determine, check, find out, see, ascertain, watch, learn (find out, learn, or determine with certainty, usually by making an inquiry or other effort) “I want to see whether she speaks French”; “See whether it works”; “find out if he speaks Russian”; “Check whether the train leaves on time”
Fig. 2.7. WordNet entry for the word“learning”.The entry shows two senses for learning as a noun
and six as verb. Synonyms and a sample sentences is given for each entry.
concepts. Synsets and relations can be accessed and navigated via the provided ap- plication programming interface. In 2010, WordNet contains in total 155,287 unique noun, verb, adjective, and adverb strings used in 206,941 word-sense pair. WordNet provides an application programming interface. WordNet captures general English and has no focus on a specific domain. Much of the vocabulary used in biology is not contained. Bodenreider et al. (2003) evaluated WordNet and found that in particular gene product symbols and cellular components are missing.
Machine learning used for synonym discovery
Turney et al. (2003) scores synonymy using web search results and evaluates the ap- proach against TOEFL questions. In the experiment a precision of 0.98 was reached. Drawback of the approach is its computational expensiveness, which make it only suitable for the validation of candidate synonyms and is not a solution for finding synonyms by scanning all possible pairings.
Recently Turney (2008) proposed to unify the algorithms for the recognition of analogies, synonyms, antonyms and associations, which have been treated indepen- dently in the past. The supervised learning algorithm is trained for each word pair, e.g. (term, synonym). The elements of the feature vectors are based on the frequen- cies of automatically defined patterns in a 250 GB text corpus of web documents. It became evident, that the accuracy of a methodology for finding synonyms corre- sponds to the size of the corpus used to obtain e.g. pairwise co-occurrences of terms for methods relying on Pointwise Mutual Information or for the learning of patterns of synonymy.
The experiment involved the answering the 80 TOEFL questions. Of 15 previously published results on this task 8 algorithms have higher and 7 lower results as the 76.2% achieved by the joined algorithm compared to 64.5% correct answers for an average applicant to a US university.
Experiment Accuracy Best previous Human Baseline
TOEFL Synonyms 0.76 0.98 0.65 0.25
Machine learning techniques have been further studied in the context of syn- onym acquisition from text. Different metrics such as Cosine Similarity, Euclidian Distance, Jaccard Index, Manhatten Distance, Jensen Shannon Divergence, and skew divergence have been compared to machine learning (Shimizu et al., 2008). For the best metric, here a machine learning classifier, a average mean precision of 0.19 was obtained. All other metrics perform even worse.
Patterns used for synonym discovery
Mccrae and Collier (2008) reported for a small scale experiment on learning regular expression patterns for synonymy a low recall of 0.30 at a precision of 0.73 which
corresponds to a study in Shimohata and Sumita (2005) reaching 0.21−0.27 coverage
at a precision greater than 0.7. When checking against WordNet precision was 1 but recall only 0.07. This shows the low coverage of WordNet, which only contained a few requested synsets. The experiment checking the UMLS showed over 0.4 recall at a precision of 0.9.
Hagiwara et al. (2006) investigated the usefulness of word relations, such as sen- tence co-occurrence, dependency, and proximity and concluded that combinations of several contextual sources lead to more stable results. Further it was experimen- tally shown, that the results become better the bigger the reference corpus is. The authors determined that word modifications are most significant between all de- pendency relationships between words. Modification in this context is defined as a limitation or qualification of on word by another word or phrase. In English lan- guage nouns or pronouns can be modified by adjectives, while adverbs can modify verbals, adjectives, and other adverbs.