Dynamic Language Models for Streaming Text

Dani Yogatama, Chong Wang, Bryan R. Routledge, Noah A. Smith, Eric P. Xing


Abstract
We present a probabilistic language model that captures temporal dynamics and conditions on arbitrary non-linguistic context features. These context features serve as important indicators of language changes that are otherwise difficult to capture using text data by itself. We learn our model in an efficient online fashion that is scalable for large, streaming data. With five streaming datasets from two different genres—economics news articles and social media—we evaluate our model on the task of sequential language modeling. Our model consistently outperforms competing models.
Anthology ID:
Q14-1015
Volume:
Transactions of the Association for Computational Linguistics, Volume 2
Month:
Year:
2014
Address:
Cambridge, MA
Editors:
Dekang Lin, Michael Collins, Lillian Lee
Venue:
TACL
SIG:
Publisher:
MIT Press
Note:
Pages:
181–192
Language:
URL:
https://aclanthology.org/Q14-1015
DOI:
10.1162/tacl_a_00175
Bibkey:
Cite (ACL):
Dani Yogatama, Chong Wang, Bryan R. Routledge, Noah A. Smith, and Eric P. Xing. 2014. Dynamic Language Models for Streaming Text. Transactions of the Association for Computational Linguistics, 2:181–192.
Cite (Informal):
Dynamic Language Models for Streaming Text (Yogatama et al., TACL 2014)
Copy Citation:
PDF:
https://aclanthology.org/Q14-1015.pdf