End-to-End Speech Recognition and Disfluency Removal

Paria Jamshid Lou, Mark Johnson


Abstract
Disfluency detection is usually an intermediate step between an automatic speech recognition (ASR) system and a downstream task. By contrast, this paper aims to investigate the task of end-to-end speech recognition and disfluency removal. We specifically explore whether it is possible to train an ASR model to directly map disfluent speech into fluent transcripts, without relying on a separate disfluency detection model. We show that end-to-end models do learn to directly generate fluent transcripts; however, their performance is slightly worse than a baseline pipeline approach consisting of an ASR system and a specialized disfluency detection model. We also propose two new metrics for evaluating integrated ASR and disfluency removal models. The findings of this paper can serve as a benchmark for further research on the task of end-to-end speech recognition and disfluency removal in the future.
Anthology ID:
2020.findings-emnlp.186
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2020
Month:
November
Year:
2020
Address:
Online
Editors:
Trevor Cohn, Yulan He, Yang Liu
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2051–2061
Language:
URL:
https://aclanthology.org/2020.findings-emnlp.186
DOI:
10.18653/v1/2020.findings-emnlp.186
Bibkey:
Cite (ACL):
Paria Jamshid Lou and Mark Johnson. 2020. End-to-End Speech Recognition and Disfluency Removal. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2051–2061, Online. Association for Computational Linguistics.
Cite (Informal):
End-to-End Speech Recognition and Disfluency Removal (Jamshid Lou & Johnson, Findings 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.findings-emnlp.186.pdf
Code
 pariajm/e2e-asr-and-disfluency-removal-evaluator +  additional community code