Deep Learning on Graphs for Natural Language Processing

Lingfei Wu, Yu Chen, Heng Ji, Yunyao Li


Abstract
Due to its great power in modeling non-Euclidean data like graphs or manifolds, deep learning on graph techniques (i.e., Graph Neural Networks (GNNs)) have opened a new door to solving challenging graph-related NLP problems. There has seen a surge of interests in applying deep learning on graph techniques to NLP, and has achieved considerable success in many NLP tasks, ranging from classification tasks like sentence classification, semantic role labeling and relation extraction, to generation tasks like machine translation, question generation and summarization. Despite these successes, deep learning on graphs for NLP still face many challenges, including automatically transforming original text sequence data into highly graph-structured data, and effectively modeling complex data that involves mapping between graph-based inputs and other highly structured output data such as sequences, trees, and graph data with multi-types in both nodes and edges. This tutorial will cover relevant and interesting topics on applying deep learning on graph techniques to NLP, including automatic graph construction for NLP, graph representation learning for NLP, advanced GNN based models (e.g., graph2seq, graph2tree, and graph2graph) for NLP, and the applications of GNNs in various NLP tasks (e.g., machine translation, natural language generation, information extraction and semantic parsing). In addition, hands-on demonstration sessions will be included to help the audience gain practical experience on applying GNNs to solve challenging NLP problems using our recently developed open source library – Graph4NLP, the first library for researchers and practitioners for easy use of GNNs for various NLP tasks.
Anthology ID:
2021.naacl-tutorials.3
Volume:
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Tutorials
Month:
June
Year:
2021
Address:
Online
Editors:
Greg Kondrak, Kalina Bontcheva, Dan Gillick
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
11–14
Language:
URL:
https://aclanthology.org/2021.naacl-tutorials.3
DOI:
10.18653/v1/2021.naacl-tutorials.3
Bibkey:
Cite (ACL):
Lingfei Wu, Yu Chen, Heng Ji, and Yunyao Li. 2021. Deep Learning on Graphs for Natural Language Processing. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Tutorials, pages 11–14, Online. Association for Computational Linguistics.
Cite (Informal):
Deep Learning on Graphs for Natural Language Processing (Wu et al., NAACL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.naacl-tutorials.3.pdf
Video:
 https://aclanthology.org/2021.naacl-tutorials.3.mp4