IIT (BHU) System for Indo-Aryan Language Identification (ILI) at VarDial 2018

Divyanshu Gupta, Gourav Dhakad, Jayprakash Gupta, Anil Kumar Singh


Abstract
Text language Identification is a Natural Language Processing task of identifying and recognizing a given language out of many different languages from a piece of text. This paper describes our submission to the ILI 2018 shared-task, which includes the identification of 5 closely related Indo-Aryan languages. We developed a word-level LSTM(Long Short-term Memory) model, a specific type of Recurrent Neural Network model, for this task. Given a sentence, our model embeds each word of the sentence and convert into its trainable word embedding, feeds them into our LSTM network and finally predict the language. We obtained an F1 macro score of 0.836, ranking 5th in the task.
Anthology ID:
W18-3921
Volume:
Proceedings of the Fifth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2018)
Month:
August
Year:
2018
Address:
Santa Fe, New Mexico, USA
Editors:
Marcos Zampieri, Preslav Nakov, Nikola Ljubešić, Jörg Tiedemann, Shervin Malmasi, Ahmed Ali
Venue:
VarDial
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
185–190
Language:
URL:
https://aclanthology.org/W18-3921
DOI:
Bibkey:
Cite (ACL):
Divyanshu Gupta, Gourav Dhakad, Jayprakash Gupta, and Anil Kumar Singh. 2018. IIT (BHU) System for Indo-Aryan Language Identification (ILI) at VarDial 2018. In Proceedings of the Fifth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2018), pages 185–190, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
Cite (Informal):
IIT (BHU) System for Indo-Aryan Language Identification (ILI) at VarDial 2018 (Gupta et al., VarDial 2018)
Copy Citation:
PDF:
https://aclanthology.org/W18-3921.pdf