Towards Lossless Encoding of Sentences

Gabriele Prato, Mathieu Duchesneau, Sarath Chandar, Alain Tapp


Abstract
A lot of work has been done in the field of image compression via machine learning, but not much attention has been given to the compression of natural language. Compressing text into lossless representations while making features easily retrievable is not a trivial task, yet has huge benefits. Most methods designed to produce feature rich sentence embeddings focus solely on performing well on downstream tasks and are unable to properly reconstruct the original sequence from the learned embedding. In this work, we propose a near lossless method for encoding long sequences of texts as well as all of their sub-sequences into feature rich representations. We test our method on sentiment analysis and show good performance across all sub-sentence and sentence embeddings.
Anthology ID:
P19-1153
Volume:
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2019
Address:
Florence, Italy
Editors:
Anna Korhonen, David Traum, Lluís Màrquez
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1577–1583
Language:
URL:
https://aclanthology.org/P19-1153
DOI:
10.18653/v1/P19-1153
Bibkey:
Cite (ACL):
Gabriele Prato, Mathieu Duchesneau, Sarath Chandar, and Alain Tapp. 2019. Towards Lossless Encoding of Sentences. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1577–1583, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Towards Lossless Encoding of Sentences (Prato et al., ACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/P19-1153.pdf
Code
 pratogab/rae
Data
BookCorpusSSTSST-2SST-5