Layout-Aware Text Representations Harm Clustering Documents by Type

Catherine Finegan-Dollak, Ashish Verma


Abstract
Clustering documents by type—grouping invoices with invoices and articles with articles—is a desirable first step for organizing large collections of document scans. Humans approaching this task use both the semantics of the text and the document layout to assist in grouping like documents. LayoutLM (Xu et al., 2019), a layout-aware transformer built on top of BERT with state-of-the-art performance on document-type classification, could reasonably be expected to outperform regular BERT (Devlin et al., 2018) for document-type clustering. However, we find experimentally that BERT significantly outperforms LayoutLM on this task (p <0.001). We analyze clusters to show where layout awareness is an asset and where it is a liability.
Anthology ID:
2020.insights-1.9
Volume:
Proceedings of the First Workshop on Insights from Negative Results in NLP
Month:
November
Year:
2020
Address:
Online
Editors:
Anna Rogers, João Sedoc, Anna Rumshisky
Venue:
insights
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
60–65
Language:
URL:
https://aclanthology.org/2020.insights-1.9
DOI:
10.18653/v1/2020.insights-1.9
Bibkey:
Cite (ACL):
Catherine Finegan-Dollak and Ashish Verma. 2020. Layout-Aware Text Representations Harm Clustering Documents by Type. In Proceedings of the First Workshop on Insights from Negative Results in NLP, pages 60–65, Online. Association for Computational Linguistics.
Cite (Informal):
Layout-Aware Text Representations Harm Clustering Documents by Type (Finegan-Dollak & Verma, insights 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.insights-1.9.pdf
Video:
 https://slideslive.com/38940796
Data
RVL-CDIP