In the previous section a link graph is constructed using the results of a linking method based on Levenshtein edit distance. A number of links has been left out of the graph for testing. However, testing on this set could suffer from a bias caused by the linking method. The link graph might be able to predict additional links based on Levenshtein distance, but this does not imply that links based on other methods can be predicted. This type of links is however the most interesting to predict, because these additional links are not yet discovered. Therefore, testing on a set of links created with a different linking method has been performed to improve the analysis of the value of the predictions.
An alternative linking method is based onstemming. For every name the stem is extracted, and records with the same combination of name stems are assumed to be linked. The name stem is a representation of the meaning (i.e., semantic origin) of the name, which results in a different type of similarity computation as compared to edit distance: two names with a large edit distance but with the same meaning are linked, while two names with a small edit distance but a different meaning are rejected. As an example, consider the marriage of Pieter Redeker and Albertje Boswijk in 1835.
This certificate is linked to a certificate from 1863 containing the parent couplePieter
RedekerandAlbertje Bos. The edit distance betweenBoswijkandBosis 4 (deletion of
wijk), which is in general not acceptable for a match. However, the stem of both names is the same (Bos, English: forest). Therefore, the stemming algorithm suggests a link. Additional information from the data set (municipality, time period, birth certificates) confirms the link. In the current experiment stemming is used for family names only.
Stemming procedure Morphological analysis using manually or automatically de- rived rules can be used for highly accuracte stemming of natural language texts in vari- ous languages such as English or Dutch [100, 17, 111]. For highly inflectional languages such as Polish or Russian grammatical declension patterns can be used to perform stem- ming [97, 101]. However, rule-based approaches have limitations for person names [95]. Names are not subject to standardized spelling conventions, which increases the amount of variation. Many names occur with very low frequencies which makes it difficult to obtain a training corpus with a sufficient level of coverage. The majority of names differ from their stem, unlike standard text. An alternative to morphology-based stemming that is less sensitive to these difficulties is lexicon-based stemming, which does not re- quire any training data. The lexicon can be used for look-up after application of suffix stripping rules [121]. However, in the current approach suffix stripping is not used. Alternatively, the stem of a name is defined as a lexicon item that forms a prefix of a name.
Family names are good candidates for lexicon-based stemming. The basis of a family name in most European languages is generally an element from one out of four categories [83]: patronyms derived from first names (Jansen, English: Johnson), loca- tion names (Bos), profession (Molenaar, English:Miller) or personal characteristic (De Jong, English:Young). The Genlias data set contains many first names and town names, which can serve as a lexicon for the first two categories. Professions, personal charac- teristics and locations other than town names are extracted from Opentaal3, which is a
general purpose word list for Dutch. Opentaal is a distributed computing project for au- tomatically harvesting Dutch words on web pages. The resulting word list is manually checked and approved by the Dutch language authority (Taalunie).
Candidate selection In some cases, there is more than one candidate for a name stem. Within a category, the candidate with the highest frequency is selected as the stem. Between categories, the longest candidate stem is selected. Frequency information for first names is extracted from the Genlias data set. For town names, the longest candidate stem is always selected. The Opentaal word list includes frequency information, which is obtained from the harvested web pages.
Frequent candidates are preferred because they generally represent the base lemma of a word. According to the stemming assumption, most links can be found using the base lemma because this lemma can be reached from a large number of variants. How- ever, the frequency approach does not work in case a short common word (either a function word or a content word) is a prefix of the base lemma. Therefore, a stop list is introduced to exclude items above a predefined frequency threshold. As an exam- ple consider the nameMandema, for which the general purpose word list contains the candidate stemsma, man, mand, mande. The candidate with the highest frequency is
man(English: idem), however this word is contained in the stop list, as isma(English: idem). From the two remaining candidatesmand(English:basket) is selected over the very archaicmande(English:community) based on frequency.
Spelling normalization In contrast to inflectional morphology, local spelling variation in names is relatively systematic. The difference between modern and archaic spelling is an important source of spelling variation [73], which can be described by a limited number of rewriting rules. In case the variation occurs in a suffix of the name, the stemming procedure is able to find a link based on the stem alone. However, if the variation occurs in the stem, rewriting can increase coverage of the linking method. Therefore in the current experiment rewriting rules from [21] are used for preprocessing of all family names. The rules are phonetically in nature, e.g.,uuy→uioreick→ek.
Linguistic and statistical properties The lexicon-based stemming method could be improved further by taking linguistic and statistical properties into account. Candidates can be selected according to grammatical category (parts-of-speech) which increases the semantic validity of the stem. The word list can be limited to 19th century vocabulary in order to increase the number of matches. The frequency of a stem in the Genlias data can
b a a b a b a b a b a b Stem No link Levenshtein removed Levenshtein
Levenshtein matches Stem matches Not matched
Figure 3.2: Records divided by link type, corresponding to the scores in Table 3.1.
be used instead of the frequency on the harvested web pages from Opentaal. However, for the present purposes it is not necessary to develop an optimal stemming procedure. The goal of the current experiments is to evaluate the use of a link graph in predicting additional links. The stemming method as described here provides a sufficient number of correct links to be able to evaluate the prediction algorithm.