Measuring the Effect of Conversational Aspects on Machine Translation Quality

Marlies van der Wees, Arianna Bisazza, Christof Monz


Abstract
Research in statistical machine translation (SMT) is largely driven by formal translation tasks, while translating informal text is much more challenging. In this paper we focus on SMT for the informal genre of dialogues, which has rarely been addressed to date. Concretely, we investigate the effect of dialogue acts, speakers, gender, and text register on SMT quality when translating fictional dialogues. We first create and release a corpus of multilingual movie dialogues annotated with these four dialogue-specific aspects. When measuring translation performance for each of these variables, we find that BLEU fluctuations between their categories are often significantly larger than randomly expected. Following this finding, we hypothesize and show that SMT of fictional dialogues benefits from adaptation towards dialogue acts and registers. Finally, we find that male speakers are harder to translate and use more vulgar language than female speakers, and that vulgarity is often not preserved during translation.
Anthology ID:
C16-1242
Volume:
Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers
Month:
December
Year:
2016
Address:
Osaka, Japan
Editors:
Yuji Matsumoto, Rashmi Prasad
Venue:
COLING
SIG:
Publisher:
The COLING 2016 Organizing Committee
Note:
Pages:
2571–2581
Language:
URL:
https://aclanthology.org/C16-1242
DOI:
Bibkey:
Cite (ACL):
Marlies van der Wees, Arianna Bisazza, and Christof Monz. 2016. Measuring the Effect of Conversational Aspects on Machine Translation Quality. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2571–2581, Osaka, Japan. The COLING 2016 Organizing Committee.
Cite (Informal):
Measuring the Effect of Conversational Aspects on Machine Translation Quality (van der Wees et al., COLING 2016)
Copy Citation:
PDF:
https://aclanthology.org/C16-1242.pdf
Data
OpenSubtitles