Optimizing Computer-Assisted Transcription Quality with Iterative User Interfaces

Matthias Sperber, Graham Neubig, Satoshi Nakamura, Alex Waibel


Abstract
Computer-assisted transcription promises high-quality speech transcription at reduced costs. This is achieved by limiting human effort to transcribing parts for which automatic transcription quality is insufficient. Our goal is to improve the human transcription quality via appropriate user interface design. We focus on iterative interfaces that allow humans to solve tasks based on an initially given suggestion, in this case an automatic transcription. We conduct a user study that reveals considerable quality gains for three variations of iterative interfaces over a non-iterative from-scratch transcription interface. Our iterative interfaces included post-editing, confidence-enhanced post-editing, and a novel retyping interface. All three yielded similar quality on average, but we found that the proposed retyping interface was less sensitive to the difficulty of the segment, and superior when the automatic transcription of the segment contained relatively many errors. An analysis using mixed-effects models allows us to quantify these and other factors and draw conclusions over which interface design should be chosen in which circumstance.
Anthology ID:
L16-1314
Volume:
Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)
Month:
May
Year:
2016
Address:
Portorož, Slovenia
Editors:
Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Sara Goggi, Marko Grobelnik, Bente Maegaard, Joseph Mariani, Helene Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
1986–1992
Language:
URL:
https://aclanthology.org/L16-1314
DOI:
Bibkey:
Cite (ACL):
Matthias Sperber, Graham Neubig, Satoshi Nakamura, and Alex Waibel. 2016. Optimizing Computer-Assisted Transcription Quality with Iterative User Interfaces. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16), pages 1986–1992, Portorož, Slovenia. European Language Resources Association (ELRA).
Cite (Informal):
Optimizing Computer-Assisted Transcription Quality with Iterative User Interfaces (Sperber et al., LREC 2016)
Copy Citation:
PDF:
https://aclanthology.org/L16-1314.pdf