Generating Image Captions in Arabic using Root-Word Based Recurrent Neural Networks and Deep Neural Networks

Vasu Jindal


Abstract
Image caption generation has gathered widespread interest in the artificial intelligence community. Automatic generation of an image description requires both computer vision and natural language processing techniques. While, there has been advanced research in the English caption generation, research on generating Arabic descriptions of an image is extremely limited. Semitic languages like Arabic are heavily influenced by root-words. We leverage this critical dependency of Arabic to generate captions of an image directly in Arabic using root-word based Recurrent Neural Network and Deep Neural Networks. Experimental results on dataset from various Middle Eastern newspaper websites allow us to report the first BLEU score for direct Arabic caption generation. We also compare the results of our approach with BLEU score captions generated in English and translated in Arabic. Experimental results confirm that generating image captions using root-words directly in Arabic significantly outperforms the English-Arabic translated captions using state-of-the-art methods.
Anthology ID:
N18-4020
Volume:
Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop
Month:
June
Year:
2018
Address:
New Orleans, Louisiana, USA
Editors:
Silvio Ricardo Cordeiro, Shereen Oraby, Umashanthi Pavalanathan, Kyeongmin Rim
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
144–151
Language:
URL:
https://aclanthology.org/N18-4020
DOI:
10.18653/v1/N18-4020
Bibkey:
Cite (ACL):
Vasu Jindal. 2018. Generating Image Captions in Arabic using Root-Word Based Recurrent Neural Networks and Deep Neural Networks. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop, pages 144–151, New Orleans, Louisiana, USA. Association for Computational Linguistics.
Cite (Informal):
Generating Image Captions in Arabic using Root-Word Based Recurrent Neural Networks and Deep Neural Networks (Jindal, NAACL 2018)
Copy Citation:
PDF:
https://aclanthology.org/N18-4020.pdf
Data
ImageNet