AIN: Fast and Accurate Sequence Labeling with Approximate Inference Network

Xinyu Wang, Yong Jiang, Nguyen Bach, Tao Wang, Zhongqiang Huang, Fei Huang, Kewei Tu


Abstract
The linear-chain Conditional Random Field (CRF) model is one of the most widely-used neural sequence labeling approaches. Exact probabilistic inference algorithms such as the forward-backward and Viterbi algorithms are typically applied in training and prediction stages of the CRF model. However, these algorithms require sequential computation that makes parallelization impossible. In this paper, we propose to employ a parallelizable approximate variational inference algorithm for the CRF model. Based on this algorithm, we design an approximate inference network that can be connected with the encoder of the neural CRF model to form an end-to-end network, which is amenable to parallelization for faster training and prediction. The empirical results show that our proposed approaches achieve a 12.7-fold improvement in decoding speed with long sentences and a competitive accuracy compared with the traditional CRF approach.
Anthology ID:
2020.emnlp-main.485
Volume:
Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
Month:
November
Year:
2020
Address:
Online
Editors:
Bonnie Webber, Trevor Cohn, Yulan He, Yang Liu
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
6019–6026
Language:
URL:
https://aclanthology.org/2020.emnlp-main.485
DOI:
10.18653/v1/2020.emnlp-main.485
Bibkey:
Cite (ACL):
Xinyu Wang, Yong Jiang, Nguyen Bach, Tao Wang, Zhongqiang Huang, Fei Huang, and Kewei Tu. 2020. AIN: Fast and Accurate Sequence Labeling with Approximate Inference Network. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6019–6026, Online. Association for Computational Linguistics.
Cite (Informal):
AIN: Fast and Accurate Sequence Labeling with Approximate Inference Network (Wang et al., EMNLP 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.emnlp-main.485.pdf
Video:
 https://slideslive.com/38938950
Code
 Alibaba-NLP/AIN
Data
ATISCoNLLCoNLL 2003