Bag & Tag’em - A New Dutch Stemmer

Anne Jonker, Corné de Ruijt, Jornt de Gruijl


Abstract
We propose a novel stemming algorithm that is both robust and accurate compared to state-of-the-art solutions, yet addresses several of the problems that current stemmers face in the Dutch language. The main issue is that most current stemmers cannot handle 3rd person singular forms of verbs and many irregular words and conjugations, unless a (nearly) brute-force approach is used. Our algorithm combines a new tagging module with a stemmer that uses tag-specific sets of rigid rules: the Bag & Tag’em (BT) algorithm. The tagging module is developed and evaluated using three algorithms: Multinomial Logistic Regression (MLR), Neural Network (NN) and Extreme Gradient Boosting (XGB). The stemming module’s performance is compared with that of current state-of-the-art stemming algorithms for the Dutch Language. Even though there is still room for improvement, the new BT algorithm performs well in the sense that it is more accurate than the current stemmers and faster than brute-force-like algorithms. The code and data used for this paper can be found at: https://github.com/Anne-Jonker/Bag-Tag-em.
Anthology ID:
2020.lrec-1.477
Volume:
Proceedings of the Twelfth Language Resources and Evaluation Conference
Month:
May
Year:
2020
Address:
Marseille, France
Editors:
Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
3868–3876
Language:
English
URL:
https://aclanthology.org/2020.lrec-1.477
DOI:
Bibkey:
Cite (ACL):
Anne Jonker, Corné de Ruijt, and Jornt de Gruijl. 2020. Bag & Tag’em - A New Dutch Stemmer. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 3868–3876, Marseille, France. European Language Resources Association.
Cite (Informal):
Bag & Tag’em - A New Dutch Stemmer (Jonker et al., LREC 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.lrec-1.477.pdf
Code
 Anne-Jonker/Bag-Tag-em