Improved Differentiable Architecture Search for Language Modeling and Named Entity Recognition

Yufan Jiang, Chi Hu, Tong Xiao, Chunliang Zhang, Jingbo Zhu


Abstract
In this paper, we study differentiable neural architecture search (NAS) methods for natural language processing. In particular, we improve differentiable architecture search by removing the softmax-local constraint. Also, we apply differentiable NAS to named entity recognition (NER). It is the first time that differentiable NAS methods are adopted in NLP tasks other than language modeling. On both the PTB language modeling and CoNLL-2003 English NER data, our method outperforms strong baselines. It achieves a new state-of-the-art on the NER task.
Anthology ID:
D19-1367
Volume:
Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
Month:
November
Year:
2019
Address:
Hong Kong, China
Editors:
Kentaro Inui, Jing Jiang, Vincent Ng, Xiaojun Wan
Venues:
EMNLP | IJCNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
3585–3590
Language:
URL:
https://aclanthology.org/D19-1367
DOI:
10.18653/v1/D19-1367
Bibkey:
Cite (ACL):
Yufan Jiang, Chi Hu, Tong Xiao, Chunliang Zhang, and Jingbo Zhu. 2019. Improved Differentiable Architecture Search for Language Modeling and Named Entity Recognition. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3585–3590, Hong Kong, China. Association for Computational Linguistics.
Cite (Informal):
Improved Differentiable Architecture Search for Language Modeling and Named Entity Recognition (Jiang et al., EMNLP-IJCNLP 2019)
Copy Citation:
PDF:
https://aclanthology.org/D19-1367.pdf
Code
 jiangyingjunn/i-darts
Data
CoNLLCoNLL 2003PTB Diagnostic ECG Database