A Deep Network with Visual Text Composition Behavior

Hongyu Guo


Abstract
While natural languages are compositional, how state-of-the-art neural models achieve compositionality is still unclear. We propose a deep network, which not only achieves competitive accuracy for text classification, but also exhibits compositional behavior. That is, while creating hierarchical representations of a piece of text, such as a sentence, the lower layers of the network distribute their layer-specific attention weights to individual words. In contrast, the higher layers compose meaningful phrases and clauses, whose lengths increase as the networks get deeper until fully composing the sentence.
Anthology ID:
P17-2059
Volume:
Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
Month:
July
Year:
2017
Address:
Vancouver, Canada
Editors:
Regina Barzilay, Min-Yen Kan
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
372–377
Language:
URL:
https://aclanthology.org/P17-2059
DOI:
10.18653/v1/P17-2059
Bibkey:
Cite (ACL):
Hongyu Guo. 2017. A Deep Network with Visual Text Composition Behavior. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 372–377, Vancouver, Canada. Association for Computational Linguistics.
Cite (Informal):
A Deep Network with Visual Text Composition Behavior (Guo, ACL 2017)
Copy Citation:
PDF:
https://aclanthology.org/P17-2059.pdf