Los objetivos impulsados

Phrase Common Uniq. Align. Uniq. Align. Types to Both in 1st type in 2nd type STR & CON 161,314 1,983,162 501,822 STR & DEP 144,834 1,999,642 438,698 STR & PERC 143,162 2,001,314 421,850 CON& DEP 399,220 263,916 184,312 CON& PERC 497,159 165,977 67,853 DEP& PERC 376,377 207,155 188,635

Table 2.5: Number of common and unique alignments (phrase pairs) for each method: Europarl data

The total number of entries in each of the four phrase tables (Europarl data) are STR: 2,144,476, CON: 663,136, DEP: 583,532, and PERC: 565,012. We can see that the CONt-table is just 31% of the size of the full STR t-table, with DEPjust 27% and PERC even smaller at just 26% of the size. By correlating the t-table7 sizes and the system performance in Table 2.3 of the four base systems,8 _{it can be concluded that the much}

smaller pure syntactic systems (CON, DEP, and PERC) give a high-quality yield.

Table 2.5 compares pairs of phrase tables and displays the overlap as well as unique phrase pairs extracted under each of the four methods. It is interesting that despite the huge size of the STRphrase table, there is very little overlap with any of the other methods; the largest overlap with STRis using CONphrases, but this amounts to only 7.5% of the STR phrase table derived via CON, and only 24% of the CON phrase table derived via STR.

The largest overlap in pure numerical terms is between CON and PERC; 75% of the CON phrase table is common with PERC, whereas 88% of the PERC phrase pairs are

7_{T-table or translation table is another term for phrase table.} 8_{These are STR, CON, DEPand PERCsystems.}

common with CON. Given that the PERCphrases are derived directly from the CONtrees, one might have expected these two to have the biggest intersection. However, surprisingly, the output (translated sentences produced by CONand PERCsystems) has a 30% overlap only. Therefore, it seems that despite a huge overlap in the phrase table configurations, the systems are different enough to produce different translations. We leave for future work an investigation into any bias here.9 _{We also acknowledge the fact that different}

parsers could jointly produce even more new phrase pairs.

Note that the overlap numbers (column 2 (Common to Both) from Table 2.5) refer to identical phrase pairs in both phrase tables under consideration, i.e. overlap on both the source language (French) and the target language (English) side. The remaining phrase pairs in each of the two phrase tables under consideration (Unique Alignments in each type: columns 3 and 4 from Table 2.5) are bound to have phrase pairs wherein there is an overlap on the source language side but not on the target language side, and vice versa. We perform a deeper investigation into such one-sided overlaps between the six pairs of phrase tables and report in Table 2.6 on the number of phrase pairs10which have a common source language phrase aligned to a dissimilar target language phrase, i.e. overlap on the source side only. This will demonstrate whether we are extending existing phrases in one phrase table with additional translations from another phrase table.

Phrase Ext. Src. BLEU IMP. Ext. Src. BLEU IMP. Types Alig. in 1st type Over 2nd Alig. in 2nd type Over 1st

STR& CON 143,317 3.86 211,203 1.0 STR& DEP 141,112 4.06 203,363 0.80 STR& PERC 132,607 3.58 189,189 0.95 CON& DEP 88,706 1.08 80,279 0.68 CON& PERC 67,747 0.50 50,102 0.73 DEP& PERC 73,052 0.70 66,873 1.33

Table 2.6: Number of extended phrase pairs (overlap on source side only) and BLEU score improvements for combined system over single system for each method: Europarl data

Table 2.6 gives the number of phrase pairs that overlap on the source language side in

9_{Specific details are covered in Chapter 5, Section 5.3.1.}

10_{Note the phrase pairs in Table 2.6 (Extended Source Alignments in each type: columns 2 and 4) are a}

a pairwise comparison of the four types of phrase tables. The BLEUIMP. (columns 3 and 5) are displayed to verify the impact of the extended phrases in terms of BLEUevaluation metric. As evident, there is a direct correlation between the number of unique phrase pairs on the target side and the system level performance. This is as expected – the norm being more data implies better performance.

For each of the four phrase extraction methods, the average number of phrase pairs per sentence and the highest number of phrase pairs in a sentence were computed as follows: JOC corpus– (STR: 35.37 (134), CON: 17.62 (71), DEP: 17.82 (71), PERC: 8.45 (53)) and Europarl corpus– (STR: 20.33 (45), CON: 10.82 (27), DEP: 10.67 (27), PERC: 10.66 (26)). Similar performance is seen between the three non-STR methods on Europarl, whereas on JOC our PERC model produces fewer alignments. The smaller number of phrase pair alignments might very well prove useful for systems with a smaller footprint requiring smaller t-tables (Sanchez-Martinez and Way, 2009).

STRPhrase Pairs CON Phrase Pairs

la commission ↔ the commission la commission ↔ the commission des ↔ of the le conseil ↔ the council , mais ↔ , but ce rapport ↔ this report , nous ↔ , we le rapport ↔ the report

, je ↔ , i en europe ↔ in europe

DEPPhrase Pairs PERCPhrase Pairs

la commission ↔ the commission la commission ↔ the commission le conseil ↔ the council le conseil ↔ the council ce rapport ↔ this report ce rapport ↔ this report

le rapport ↔ the report le rapport ↔ the report l’ union ↔ the union l’ ue ↔ the eu

Figure 2.6: Top 5 phrase pairs (with target length constrained to 2 words) for each of the four phrase extractions, namely STR, CON, DEP, and PERC: Europarl data

A small sample of the types of chunks produced by each of the four phrase extraction methodologies in Figure 2.6 gives a clearer picture of how STR phrase pairs differ from the linguistically motivated phrase pairs (CON, DEP, and PERC). For example, the STR t-table contains a large number of non-linguistic sequences of words, often containing punctuation marks.

important step is to evaluate whether these unique chunks are of use in PB-SMT.