M ONOGRÁFICO
2. PRIMERA ETAPA (1951-1984): LA AUSENCIA DE DEBATE IDENTITARIO Y EL INFORME TINDEMANS
Parsing models that have used the Wall Street Journal Corpus have focused on two main tasks: full syntactic parsing [5, 7, 98, 100, 121], which the parsing model presented in this work does, and Semantic Role Labelling [147, 149, 150, 151, 152]. Reported work on full syntactic parsing has mostly involved traditional statistical parsing models which continue to represent the state-of-the-art for broad coverage natural language parsing (table 5.15). While this connectionist parsing
model does not yet compare favourably with the statistical parsers in terms of performance, it has achieved its performance with a far smaller training data set size. Its training to test data set ratio is 1:5.14.
Table 5.16: Comparison of Syntactic Parser Results on the WSJ Corpus
Full
5.10 Discussion
To enable the combination of lexical semantic information with syntactic knowledge as input to the parser, an algorithm, involving a semi-automatic semantic tagging procedure, has been developed for the semantic annotation of nouns in the BLLIP WSJ corpus. A manual parse of semantically tagged sentences from the WSJ corpus shows that the noun classes obtained from WordNet provide sufficient information to aid the disambiguation of preposition attachment cases. They have also been found to be sufficient in the preposition attachment resolution for the following sentence pairs (POS tags for the first three pairs are the same; including the lexical semantic information provides useful additional knowledge):
1a) The boy ate the pasta with the sauce.
1b) The boy ate the pasta with he fork.
2a) The boy broke the window with the curtain.
2b) The boy broke the window with the rock.
3a) The policeman chased the boy with a limp.
3b) The policeman chased the boy with a truncheon.
4a) I examined the man with a stethoscope.
4b) I examined the man with a broken leg.
For all three connectionist modules of this parser, the networks trained on the combination of lexical semantic and syntactic input representation (compared to the networks trained on only syntactic input representations) dealt with more complex tasks (using a greater number of network connections) and had to be trained for longer periods in terms of number of epochs and actual training time. The delimiter networks learnt slightly lower proportions of the sequences presented to them when trained on a combination of lexical semantic and syntactic input representation, with both the LRD and RLD networks learning over 94.5% of
sequences in both input representation cases. However, a comparison of the generalisation performances of the delimiter networks, using two test sets, reveals very close performances when both sets of input representations are used. The generalisation performance of each delimiter network when lexical semantic information is added appears to be at par with its performance when only syntactic information is used.
In comparing the performances of the delimiters at sequence length level, there was a similar trend in learning performance for sequences of the same length, apart from one case. This is the case with LRD sequences of length 7, there is 9.69%
difference in training performance between both input representation instances; the LRD network with a combination of lexical semantic and syntactic input representation learns only 86.99% of these sequences. Given that LRD sequences make use of three look-back and three look-ahead symbols, sequences of length 7 would normally be phrases with only one word or constituent, for example a single noun forming a noun phrase. The introduction of additional information seems to have made the parsing of phrases like this a more difficult task. This could be solved by fitting the network better to its training examples (longer training times and more optimal networks).
For the RLD, sequences with sequence lengths of 10, 11, 12 and 13 did not perform as well as sequences of other lengths (8, 9, 14, 15, 16 and 17). Sequences with these four least performing sequence lengths are the four most frequent sequences, in terms of sequence length. The RLD sequences include six look-back
balancing issues as raised for the LRD. It is worth noting that for the RLD, sequences with the shortest sequence lengths (8 and 9) are among the high performers. Considering the presence of number of look-back symbols (6) and the end-of-sentence symbol, ‘*’, phrases involved here would usually have one or two symbols; there are not very many of these phrases.
Another similarity in results obtained for the two different set of input representations is the consistency in the generalisation result on test sets generated from sentences of very different sample sizes. Generalisation performances were about the same for test data generated from 74 sentences and for those generated from 1059 sentences. This shows that the generalisation performance in these cases is not deeply affected by test sample size.
On the whole, the module level performances of the delimiter networks seemed to be at par, irrespective of the set of input representation used. The module level performance of the phrase recognition network that used only syntactic input representation appeared to perform better than when a combination of lexical semantic and syntactic information was used. This also seemed the case with performances at the sentence level. However, when using a combination of lexical semantic and syntactic information, the sentence level results have kept improving with positive modifications to the component connectionist modules. Its F-Measure has improved from 69.26% to 73.45% and up to 73.60% with training improvement to different component connectionist modules. This indicates a lot of room for improvement of the sentence level performance if the connectionist modules are further optimised. To fully grasp the effect of combining lexical semantic and syntactic input representation on the parser, an examination of the parser’s behaviour on the two sets of input representations is necessary. This is done in the next chapter.