RDG-Map: A Multimodal Corpus of Pedagogical Human-Agent Spoken Interactions.

Maike Paetzel, Deepthi Karkada, Ramesh Manuvinakurike


Abstract
This paper presents a multimodal corpus of 209 spoken game dialogues between a human and a remote-controlled artificial agent. The interactions involve people collaborating with the agent to identify countries on the world map as quickly as possible, which allows studying rapid and spontaneous dialogue with complex anaphoras, disfluent utterances and incorrect descriptions. The corpus consists of two parts: 8 hours of game interactions have been collected with a virtual unembodied agent online and 26.8 hours have been recorded with a physically embodied robot in a research lab. In addition to spoken audio recordings available for both parts, camera recordings and skeleton-, facial expression- and eye-gaze tracking data have been collected for the lab-based part of the corpus. In this paper, we introduce the pedagogical reference resolution game (RDG-Map) and the characteristics of the corpus collected. We also present an annotation scheme we developed in order to study the dialogue strategies utilized by the players. Based on a subset of 330 minutes of interactions annotated so far, we discuss initial insights into these strategies as well as the potential of the corpus for future research.
Anthology ID:
2020.lrec-1.75
Volume:
Proceedings of the Twelfth Language Resources and Evaluation Conference
Month:
May
Year:
2020
Address:
Marseille, France
Editors:
Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
600–609
Language:
English
URL:
https://aclanthology.org/2020.lrec-1.75
DOI:
Bibkey:
Cite (ACL):
Maike Paetzel, Deepthi Karkada, and Ramesh Manuvinakurike. 2020. RDG-Map: A Multimodal Corpus of Pedagogical Human-Agent Spoken Interactions.. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 600–609, Marseille, France. European Language Resources Association.
Cite (Informal):
RDG-Map: A Multimodal Corpus of Pedagogical Human-Agent Spoken Interactions. (Paetzel et al., LREC 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.lrec-1.75.pdf