Improving Sentence Boundary Detection for Spoken Language Transcripts

Ines Rehbein, Josef Ruppenhofer, Thomas Schmidt


Abstract
This paper presents experiments on sentence boundary detection in transcripts of spoken dialogues. Segmenting spoken language into sentence-like units is a challenging task, due to disfluencies, ungrammatical or fragmented structures and the lack of punctuation. In addition, one of the main bottlenecks for many NLP applications for spoken language is the small size of the training data, as the transcription and annotation of spoken language is by far more time-consuming and labour-intensive than processing written language. We therefore investigate the benefits of data expansion and transfer learning and test different ML architectures for this task. Our results show that data expansion is not straightforward and even data from the same domain does not always improve results. They also highlight the importance of modelling, i.e. of finding the best architecture and data representation for the task at hand. For the detection of boundaries in spoken language transcripts, we achieve a substantial improvement when framing the boundary detection problem assentence pair classification task, as compared to a sequence tagging approach.
Anthology ID:
2020.lrec-1.878
Volume:
Proceedings of the Twelfth Language Resources and Evaluation Conference
Month:
May
Year:
2020
Address:
Marseille, France
Editors:
Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
7102–7111
Language:
English
URL:
https://aclanthology.org/2020.lrec-1.878
DOI:
Bibkey:
Cite (ACL):
Ines Rehbein, Josef Ruppenhofer, and Thomas Schmidt. 2020. Improving Sentence Boundary Detection for Spoken Language Transcripts. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 7102–7111, Marseille, France. European Language Resources Association.
Cite (Informal):
Improving Sentence Boundary Detection for Spoken Language Transcripts (Rehbein et al., LREC 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.lrec-1.878.pdf