FLiText: A Faster and Lighter Semi-Supervised Text Classification with Convolution Networks

Chen Liu, Zhang Mengchao, Fu Zhibing, Panpan Hou, Yu Li


Abstract
In natural language processing (NLP), state-of-the-art (SOTA) semi-supervised learning (SSL) frameworks have shown great performance on deep pre-trained language models such as BERT, and are expected to significantly reduce the demand for manual labeling. However, our empirical studies indicate that these frameworks are not suitable for lightweight models such as TextCNN, LSTM and etc. In this work, we develop a new SSL framework called FLiText, which stands for Faster and Lighter semi-supervised Text classification. FLiText introduces an inspirer network together with the consistency regularization framework, which leverages a generalized regular constraint on the lightweight models for efficient SSL. As a result, FLiText obtains new SOTA performance for lightweight models across multiple SSL benchmarks on text classification. Compared with existing SOTA SSL methods on TextCNN, FLiText improves the accuracy of lightweight model TextCNN from 51.00% to 90.49% on IMDb, 39.8% to 58.06% on Yelp-5, and from 55.3% to 65.08% on Yahoo! Answer. In addition, compared with the fully supervised method on the full dataset, FLiText just uses less than 1% of labeled data to improve the accuracy by 6.59%, 3.94%, and 3.22% on the datasets of IMDb, Yelp-5, and Yahoo! Answer respectively.
Anthology ID:
2021.emnlp-main.192
Volume:
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2021
Address:
Online and Punta Cana, Dominican Republic
Editors:
Marie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-tau Yih
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2481–2491
Language:
URL:
https://aclanthology.org/2021.emnlp-main.192
DOI:
10.18653/v1/2021.emnlp-main.192
Bibkey:
Cite (ACL):
Chen Liu, Zhang Mengchao, Fu Zhibing, Panpan Hou, and Yu Li. 2021. FLiText: A Faster and Lighter Semi-Supervised Text Classification with Convolution Networks. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2481–2491, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
FLiText: A Faster and Lighter Semi-Supervised Text Classification with Convolution Networks (Liu et al., EMNLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.emnlp-main.192.pdf
Video:
 https://aclanthology.org/2021.emnlp-main.192.mp4
Code
 valuesimplex/flitext
Data
IMDb Movie Reviews