Language Models as an Alternative Evaluator of Word Order Hypotheses: A Case Study in Japanese

Tatsuki Kuribayashi, Takumi Ito, Jun Suzuki, Kentaro Inui


Abstract
We examine a methodology using neural language models (LMs) for analyzing the word order of language. This LM-based method has the potential to overcome the difficulties existing methods face, such as the propagation of preprocessor errors in count-based methods. In this study, we explore whether the LM-based method is valid for analyzing the word order. As a case study, this study focuses on Japanese due to its complex and flexible word order. To validate the LM-based method, we test (i) parallels between LMs and human word order preference, and (ii) consistency of the results obtained using the LM-based method with previous linguistic studies. Through our experiments, we tentatively conclude that LMs display sufficient word order knowledge for usage as an analysis tool. Finally, using the LM-based method, we demonstrate the relationship between the canonical word order and topicalization, which had yet to be analyzed by large-scale experiments.
Anthology ID:
2020.acl-main.47
Original:
2020.acl-main.47v1
Version 2:
2020.acl-main.47v2
Volume:
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2020
Address:
Online
Editors:
Dan Jurafsky, Joyce Chai, Natalie Schluter, Joel Tetreault
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
488–504
Language:
URL:
https://aclanthology.org/2020.acl-main.47
DOI:
10.18653/v1/2020.acl-main.47
Bibkey:
Cite (ACL):
Tatsuki Kuribayashi, Takumi Ito, Jun Suzuki, and Kentaro Inui. 2020. Language Models as an Alternative Evaluator of Word Order Hypotheses: A Case Study in Japanese. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 488–504, Online. Association for Computational Linguistics.
Cite (Informal):
Language Models as an Alternative Evaluator of Word Order Hypotheses: A Case Study in Japanese (Kuribayashi et al., ACL 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.acl-main.47.pdf
Video:
 http://slideslive.com/38928812
Code
 kuribayashi4/LM_as_Word_Order_Evaluator