Learning Web-based Procedures by Reasoning over Explanations and Demonstrations in Context

Shashank Srivastava, Oleksandr Polozov, Nebojsa Jojic, Christopher Meek


Abstract
We explore learning web-based tasks from a human teacher through natural language explanations and a single demonstration. Our approach investigates a new direction for semantic parsing that models explaining a demonstration in a context, rather than mapping explanations to demonstrations. By leveraging the idea of inverse semantics from program synthesis to reason backwards from observed demonstrations, we ensure that all considered interpretations are consistent with executable actions in any context, thus simplifying the problem of search over logical forms. We present a dataset of explanations paired with demonstrations for web-based tasks. Our methods show better task completion rates than a supervised semantic parsing baseline (40% relative improvement on average), and are competitive with simple exploration-and-demonstration based methods, while requiring no exploration of the environment. In learning to align explanations with demonstrations, basic properties of natural language syntax emerge as learned behavior. This is an interesting example of pragmatic language acquisition without any linguistic annotation.
Anthology ID:
2020.acl-main.684
Volume:
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2020
Address:
Online
Editors:
Dan Jurafsky, Joyce Chai, Natalie Schluter, Joel Tetreault
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
7652–7662
Language:
URL:
https://aclanthology.org/2020.acl-main.684
DOI:
10.18653/v1/2020.acl-main.684
Bibkey:
Cite (ACL):
Shashank Srivastava, Oleksandr Polozov, Nebojsa Jojic, and Christopher Meek. 2020. Learning Web-based Procedures by Reasoning over Explanations and Demonstrations in Context. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7652–7662, Online. Association for Computational Linguistics.
Cite (Informal):
Learning Web-based Procedures by Reasoning over Explanations and Demonstrations in Context (Srivastava et al., ACL 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.acl-main.684.pdf
Video:
 http://slideslive.com/38929361