A brief analysis of the results for freely available machine translation systems on the Internet demonstrates that the translation of multiword expressions is currently a problem area for any machine translation system, whether statistical or rule-based. The
EVALUATION
146
ambiguity of these expressions can lead to loss of meaning, mistakes, unclear or unsuitable translations, when there is no linguistic (syntactic-semantic) knowledge associated with them. Translation results extracted from METRA prove that machine translation engines are unsuccessful at handling the translation of support verb constructions. A literal and unnatural translation is provided by some machines. Table 6 below, illustrates how machine translation engines handle translation of the Portuguese support verb construction tomar uma decisão (make a decision) in the sentence I can't make a decision about anything these days [COMPARA]. For example, for some translation engines, the English support verb construction make a decision is translated into Portuguese as fazer uma decisão instead of tomar uma decisão or even as the strong verb decidir, which represent its optimal paraphrase. This inaccuracy means that the English support verb make is directly translated into the Portuguese support verb fazer (default translation), instead of being recognized as part of the support verb construction which embeds semantic meaning as a whole.
Translation Engine Translation Results
Amikai que eu não posso fazer para uma decisão sobre qualquer coisa estes dias.
FreeTranslation Eu não posso tomar uma decisão sobre algo hoje em dia.
WorldLingo Eu no posso fazer a uma deciso sobre qualquer coisa estes dias.
E-Translation Server Não posso tomar uma decisão sobre qualquer coisa estes dias.
Table 6: METRA search results for machine translation of the English support verb construction make a decision into Portuguese (01/11/2008)
Similar problems are shown in the translations of Portuguese support verb construction tomar uma decisão into English, where the support verb tomar is being translated literally as take. This is not an incorrect translation, as the British National Corpus (BNC) presents about 6% of the examples with a/the decision occurring with take. However, the 40% results with make show that take is not the preferred support verb to go with the predicate noun decision. Machine translation is not addressing the appropriateness or suitability issue effectively.
We have tried to replace some support verb constructions with verbs and verified that overall machine translation engines showed significantly better results. For example,
EVALUATION
147 machine translation engines are unanimous in choosing the Portuguese verb decidir as the correct translation for the English verb decide, as Table 7shows.
Translation Engine Translation Results
Amikai que eu não posso decidir sobre nada estes dias.
FreeTranslation Eu não posso decidir sobre algo hoje em dia.
WorldLingo Eu no posso decidir-se sobre qualquer coisa estes dias.
E-Translation Server Não posso decidir sobre nada estes dias.
Table 7: METRA search results for machine translation of the English verb decide into Portuguese (01/11/2008)
This pre-editing, or controlled language writing by paraphrasing, improves translation results and makes output sentences more comprehensible overall. This proves that, if we consider pre-editing of the input sentences where support verb constructions occur, changing each instance into a verb (whenever that is linguistically suitable), we are not changing the meaning of the source sentence and we are giving the machine translation engine a distinctly better chance of improving the output result, by filtering out some noise, i.e., the weak verb. The support verb construction make a decision is a stylistic alternative to the verb decide, where neither the support verb make nor the determiner a adds any meaning to the expression. In fact, in support verb constructions, the support verb is often void of meaning. Trying to translate them brings additional difficulties to machine translation systems, which is unnecessary until/unless they become more sophisticated. Our paraphrasing system allows several possibilities. However, paraphrasing by simplification proved to be the most suitable for publicly available machine translation systems. The ideal framework is that machine translation systems are not limited to one possibility, as long as they translate with precision. Besides, we believe that it is pointless to challenge one limited machine translation system with structures that we know a priori this system cannot translate well. For equivalent paraphrasing, the support verb must be recognized as part of a support verb construction which must be considered as a single semantic unit. The default assumption of all machine translation systems which cannot discern whether a word, in this case a support verb, adds semantic meaning to a phrase, is to assign equal semantic value to each word individually, unless otherwise instructed. The system fails by incorrectly assigning semantic value to a support
EVALUATION
148
verb, resulting in a loss in paraphrasing capability of the output sentence. This is the problem of direct translation.
The ambiguity that support verb constructions can carry can lead to loss of meaning, mistakes and unclear translations, when there is no syntactic-semantic information associated to them. Again, we used METRA to translate unedited sentences extracted from a corpus of support verb plus biomedical-related term combinations found on the Internet and analyzed the outputs. After that, we pre-edited those sentences including subject and object whenever needed, paraphrased them accordingly and compared results. Table 8 presents an example of a sentence extracted from a less formal, oral-type of text, namely from a doctor’s blog [Blog-WWH]. The results of the machine translation engines were the following for the original corpus sentence and for the paraphrased sentences:
Por falta de condições técnicas, ele foi removido para o Hospital das Clínicas, onde se fez uma amputação a nível de ombro.
FreeTranslation For lack of technical conditions, he was removed for the Hospital of the Clinics, where was done an
amputation in terms of shoulder.
WorldLingo Due to conditions techniques, it it was removed for the Hospital of the Clinics, where if the shoulder level
made an amputation.
Por falta de condições técnicas, ele foi transportado para o Hospital das Clínicas, onde os médicos amputaram o seu braço ao nível do ombro.
FreeTranslation Due to conditions techniques, it it was carried to the Hospital of the Clinics, where the doctors had
amputated its arm to the level of the shoulder.
WorldLingo For lack of technical conditions, he was transported for the Hospital of the Clinics, where the doctors
amputated his arm level with the shoulder.
Por falta de condições técnicas, ele foi transportado para o Hospital das Clínicas, onde o seu braço foi amputado ao nível do ombro.
FreeTranslation for lack of technical conditions, he was transported for the Hospital of the Clinics, where arm was
amputated level with the shoulder.
WorldLingo Due to conditions techniques, it it was carried to the Hospital of the Clinics, where its arm was amputated
to the level of the shoulder.
Table 8: Machine translation results for sentences with support verb constructions and results for pre- edited paraphrases with predicate-argument relation knowledge (26/05/2008)
In Table 8, the translation of the support verb constructions result in corresponding English support verb constructions, where the support verb was translated literally. The sentences are confusing and unclear. WordLingo assigns the event amputation to the wrong subject. This error illustrates that it is risky to translate support verb constructions
EVALUATION
149 by using machine translation. On the contrary, the paraphrasing applied to the sentence presented more comprehensible translated text when we replaced the support verb construction by a verb and added a subject and an object to the sentence. With a semantically expressed agent (médicos > doctors) and the part of the body that was affected as the result of the procedure (o seu braço > his arm), the sentence os médicos amputaram o seu braço (the doctors amputated his arm) became clearer and easier to translate. Similar good results were obtained when we replaced the support verb construction with a passive construction and specified the object (o seu braço foi amputado > his arm was amputated).
These simple word replacement operations show that a linguistic analysis of the syntactic and semantic arguments of the predicate allows the disambiguation of the support verb construction. Translating these types of support verb constructions is very difficult unless a linguistic analysis of the predicate-argument structure is performed. These results show enough evidence to argue that paraphrasing support verb constructions with other verbal constructions and a well-defined predicate-argument structure help produce better and "safer" machine translation results.
To sum up, empirical evidence shows that application of linguistic knowledge to proper handling of support verb constructions by machine translation systems or natural language processing applications is effective. We believe that our methodology leads to attainable paraphrasing translation solutions. This work demonstrates that we can create an instrument of some utility to the research community, which has good applicability in machine translation.