Causal analysis of task completion errors in spoken music retrieval interactions

Sunao Hara, Norihide Kitaoka, Kazuya Takeda


Abstract
In this paper, we analyze the causes of task completion errors in spoken dialog systems, using a decision tree with N-gram features of the dialog to detect task-incomplete dialogs. The dialog for a music retrieval task is described by a sequence of tags related to user and system utterances and behaviors. The dialogs are manually classified into two classes: completed and uncompleted music retrieval tasks. Differences in tag classification performance between the two classes are discussed. We then construct decision trees which can detect if a dialog finished with the task completed or not, using information gain criterion. Decision trees using N-grams of manual tags and automatic tags achieved 74.2% and 80.4% classification accuracy, respectively, while the tree using interaction parameters achieved an accuracy rate of 65.7%. We also discuss more details of the causality of task incompletion for spoken dialog systems using such trees.
Anthology ID:
L12-1631
Volume:
Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12)
Month:
May
Year:
2012
Address:
Istanbul, Turkey
Editors:
Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Mehmet Uğur Doğan, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
1365–1372
Language:
URL:
http://www.lrec-conf.org/proceedings/lrec2012/pdf/1059_Paper.pdf
DOI:
Bibkey:
Cite (ACL):
Sunao Hara, Norihide Kitaoka, and Kazuya Takeda. 2012. Causal analysis of task completion errors in spoken music retrieval interactions. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12), pages 1365–1372, Istanbul, Turkey. European Language Resources Association (ELRA).
Cite (Informal):
Causal analysis of task completion errors in spoken music retrieval interactions (Hara et al., LREC 2012)
Copy Citation:
PDF:
http://www.lrec-conf.org/proceedings/lrec2012/pdf/1059_Paper.pdf