On the origin of errors: a fine-grained analysis of MT and PE errors and their relationship

Publication type
P1
Publication status
Published
Authors
Daems, J, Macken, L., & Vandepitte, S.
Editor
Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Hrafn Loftsson, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk and Stelios Piperidis
Series
LREC 2014 - NINTH INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION
Pagination
62-66
Publisher
European Language Resources Association (ELRA)
Conference
9th International Conference on Language Resources and Evaluation (LREC) (Reykjavik, Iceland)
Download
(.pdf)
Project
Robot
View in Biblio
(externe link)

Abstract

In order to improve the symbiosis between machine translation (MT) system and post-editor, it is not enough to know that the output of one system is better than the output of another system. A fine-grained error analysis is needed to provide information on the type and location of errors occurring in MT and the corresponding errors occurring after post-editing (PE). This article reports on a fine-grained translation quality assessment approach which was applied to machine translated-texts and the post-edited versions of these texts, made by student post-editors. By linking each error to the corresponding source text-passage, it is possible to identify passages that were problematic in MT, but not after PE, or passages that were problematic even after PE. This method provides rich data on the origin and impact of errors, which can be used to improve post-editor training as well as machine translation systems. We present the results of a pilot experiment on the post-editing of newspaper articles and highlight the advantages of our approach.