Sharon O’Brien


2023

pdf bib
Measuring Machine Translation User Experience (MTUX): A Comparison between AttrakDiff and User Experience Questionnaire
Vicent Briva-Iglesias | Sharon O’Brien
Proceedings of the 24th Annual Conference of the European Association for Machine Translation

Perceptions and experiences of machine translation (MT) users before, during, and after their interaction with MT systems, products or services has been overlooked both in academia and in industry. Tradi-tionally, the focus has been on productivi-ty and quality, often neglecting the human factor. We propose the concept of Ma-chine Translation User Experience (MTUX) for assessing, evaluating, and getting further information about the user experiences of people interacting with MT. By conducting a human-computer in-teraction (HCI)-based study with 15 pro-fessional translators, we analyse which is the best method for measuring MTUX, and conclude by suggesting the use of the User Experience Questionnaire (UEQ). The measurement of MTUX will help eve-ry stakeholder in the MT industry - devel-opers will be able to identify pain points for the users and solve them in the devel-opment process, resulting in better MTUX and higher adoption of MT systems or products by MT users.

2020

pdf bib
Comprehension and Trust in Crises: Investigating the Impact of Machine Translation and Post-Editing
Alessandra Rossetti | Sharon O’Brien | Patrick Cadwell
Proceedings of the 22nd Annual Conference of the European Association for Machine Translation

We conducted a survey to understand the impact of machine translation and post-editing awareness on comprehension of and trust in messages disseminated to prepare the public for a weather-related crisis, i.e. flooding. The translation direction was English–Italian. Sixty-one participants—all native Italian speakers with different English proficiency levels—answered our survey. Each participant read and evaluated between three and six crisis messages using ratings and open-ended questions on comprehensibility and trust. The messages were in English and Italian. All the Italian messages had been machine translated and post-edited. Nevertheless, participants were told that only half had been post-edited, so that we could test the impact of post-editing awareness. We could not draw firm conclusions when comparing the scores for trust and comprehensibility assigned to the three types of messages—English, post-edits, and purported raw outputs. However, when scores were triangulated with open-ended answers, stronger patterns were observed, such as the impact of fluency of the translations on their comprehensibility and trustworthiness. We found correlations between comprehensibility and trustworthiness, and identified other factors influencing these aspects, such as the clarity and soundness of the messages. We conclude by outlining implications for crisis preparedness, limitations, and areas for future research.

2019

pdf bib
What is the impact of raw MT on Japanese users of Word: preliminary results of a usability study using eye-tracking
Ana Guerberof Arenas | Joss Moorkens | Sharon O’Brien
Proceedings of Machine Translation Summit XVII: Research Track

2017

pdf bib
Machine Translation as an Academic Writing Aid for Medical Practitioners
Carla Parra Escartín | Sharon O’Brien | Marie-Josée Goulet | Michel Simard
Proceedings of Machine Translation Summit XVI: Research Track

pdf bib
Usability of web-based MT post-editing environments for screen reader users
Silvia Rodríguez Vázquez | Sharon O’Brien | Dónal Fitzpatrick
Proceedings of Machine Translation Summit XVI: Commercial MT Users and Translators Track

pdf bib
The INTERACT Project and Crisis MT
Sharon O’Brien | Chao-Hong Liu | Andy Way | João Graça | André Martins | Helena Moniz | Ellie Kemp | Rebecca Petras
Proceedings of Machine Translation Summit XVI: Commercial MT Users and Translators Track

pdf bib
The Impact of MT Quality Estimation on Post-Editing Effort
Carlos Teixeira | Sharon O’Brien
Proceedings of Machine Translation Summit XVI: Commercial MT Users and Translators Track

pdf bib
A Reception Study of Machine Translated Subtitles for MOOCs
Ke Hu | Sharon O’Brien | Dorothy Kenny
Proceedings of Machine Translation Summit XVI: Commercial MT Users and Translators Track

2016

pdf bib
Evaluating the Impact of Light Post-Editing on Usability
Sheila Castilho | Sharon O’Brien
Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)

This paper discusses a methodology to measure the usability of machine translated content by end users, comparing lightly post-edited content with raw output and with the usability of source language content. The content selected consists of Online Help articles from a software company for a spreadsheet application, translated from English into German. Three groups of five users each used either the source text - the English version (EN) -, the raw MT version (DE_MT), or the light PE version (DE_PE), and were asked to carry out six tasks. Usability was measured using an eye tracker and cognitive, temporal and pragmatic measures of usability. Satisfaction was measured via a post-task questionnaire presented after the participants had completed the tasks.

2015

pdf bib
Post-Editing Evaluations: Trade-offs between Novice and Professional Participants
Joss Moorkens | Sharon O’Brien
Proceedings of the 18th Annual Conference of the European Association for Machine Translation

pdf bib
Post-Editing Evaluations: Trade-offs between Novice and Professional Participants
Joss Moorkens | Sharon O’Brien
Proceedings of the 18th Annual Conference of the European Association for Machine Translation

2014

pdf bib
Kanjingo: a mobile app for post-editing
Sharon O’Brien | Joss Moorkens | Joris Vreeke
Proceedings of the 11th Conference of the Association for Machine Translation in the Americas

We present Kanjingo, a mobile app for post-editing currently running under iOS. The App was developed using an agile methodoly at CNGL, DCU. Though it could be used for numerous scenarios, our test scenario involved the post-editing of machine translated sample content for the non-profit translation organization Translators without Borders. Feedback from a first round of user testing for English-French and English-Spanish was positive, but users also identified a number of usability issues that required improvement. These issues were addressed in a second development round and a second usability evaluation was carried out in collaboration with another non-profit translation organization, The Rosetta Foundation, again with French and Spanish as target languages.

pdf bib
Kanjingo – a mobile app for post-editing
Sharon O’Brien | Joss Moorkens | Joris Vreeke
Proceedings of the 17th Annual Conference of the European Association for Machine Translation

pdf bib
Does post-editing increase usability? A study with Brazilian Portuguese as target language
Sheila Castilho | Sharon O’Brien | Fabio Alves | Morgan O’Brien
Proceedings of the 17th Annual Conference of the European Association for Machine Translation

pdf bib
Proceedings of the EACL 2014 Workshop on Humans and Computer-assisted Translation
Ulrich Germann | Michael Carl | Philipp Koehn | Germán Sanchis-Trilles | Francisco Casacuberta | Robin Hill | Sharon O’Brien
Proceedings of the EACL 2014 Workshop on Humans and Computer-assisted Translation

2013

pdf bib
User attitudes to the post-editing interface
Joss Moorkens | Sharon O’Brien
Proceedings of the 2nd Workshop on Post-editing Technology and Practice

pdf bib
Community-based post-editing of machine-translated content: monolingual vs. bilingual
Linda Mitchell | Johann Roturier | Sharon O’Brien
Proceedings of the 2nd Workshop on Post-editing Technology and Practice

2012

pdf bib
A User-Based Usability Assessment of Raw Machine Translated Technical Instructions
Stephen Doherty | Sharon O’Brien
Proceedings of the 10th Conference of the Association for Machine Translation in the Americas: Commercial MT User Program

This paper reports on a project whose aims are to investigate the usability of raw machine translated technical support documentation for a commercial online file storage service. Following the ISO/TR 16982 definition of usability - goal completion, satisfaction, effectiveness, and efficiency - comparisons are drawn for all measures between the original user documentation written in English for a well-known online file storage service and raw machine translated output in four target languages: Spanish, French, German and Japanese. Using native speakers for each language, we found significant differences between the source and MT output for three out of the four measures: goal completion, efficiency and user satisfaction. This leads to a tentative conclusion that there is a difference in usability between well-formed content and raw machine translated content, and we suggest avenues for further work.

2010

pdf bib
Next generation translation and localization: users are taking charge
Sharon O’Brien | Reinhard Schäler
Proceedings of Translating and the Computer 32

pdf bib
Analysing Post-Editing Performance: Correlations with Years of Translation Experience
Giselle de Almeida | Sharon O’Brien
Proceedings of the 14th Annual Conference of the European Association for Machine Translation

pdf bib
A Novel Statistical Pre-Processing Model for Rule-Based Machine Translation System
Yanli Sun | Sharon O’Brien | Minako O’Hagan | Fred Hollowood
Proceedings of the 14th Annual Conference of the European Association for Machine Translation

pdf bib
Introduction to Post-Editing: Who, What, How and Where to Next?
Sharon O’Brien
Proceedings of the 9th Conference of the Association for Machine Translation in the Americas: Tutorials

2009

pdf bib
Can MT Output Be Evaluated Through Eye Tracking?
Stephen Doherty | Sharon O’Brien
Proceedings of Machine Translation Summit XII: Posters

bib
Postediting Machine Translation Output Guidelines
Sharon O’Brien | Johann Roturier | Giselle de Almeida
Proceedings of Machine Translation Summit XII: Tutorials

2007

pdf bib
How portable are controlled language rules? A comparison of two empirical MT studies
Sharon O’Brien | Johann Roturier
Proceedings of Machine Translation Summit XI: Papers

2004

pdf bib
Machine Translatability and Post-Editing Effort: How do they relate
Sharon O’Brien
Proceedings of Translating and the Computer 26

2003

pdf bib
Controlling controlled English
Sharon O’Brien
EAMT Workshop: Improving MT through other language technology tools: resources and tools for building MT

2002

pdf bib
Teaching post-editing: a proposal for course content
Sharon O’Brien
Proceedings of the 6th EAMT Workshop: Teaching Machine Translation

1999

pdf bib
Translation Technology - The Next Generation
Sharon O’Brien
Proceedings of Translating and the Computer 21