Understanding Advertisements with BERT

Kanika Kalra, Bhargav Kurma, Silpa Vadakkeeveetil Sreelatha, Manasi Patwardhan, Shirish Karande


Abstract
We consider a task based on CVPR 2018 challenge dataset on advertisement (Ad) understanding. The task involves detecting the viewer’s interpretation of an Ad image captured as text. Recent results have shown that the embedded scene-text in the image holds a vital cue for this task. Motivated by this, we fine-tune the base BERT model for a sentence-pair classification task. Despite utilizing the scene-text as the only source of visual information, we could achieve a hit-or-miss accuracy of 84.95% on the challenge test data. To enable BERT to process other visual information, we append image captions to the scene-text. This achieves an accuracy of 89.69%, which is an improvement of 4.7%. This is the best reported result for this task.
Anthology ID:
2020.acl-main.674
Volume:
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2020
Address:
Online
Editors:
Dan Jurafsky, Joyce Chai, Natalie Schluter, Joel Tetreault
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
7542–7547
Language:
URL:
https://aclanthology.org/2020.acl-main.674
DOI:
10.18653/v1/2020.acl-main.674
Bibkey:
Cite (ACL):
Kanika Kalra, Bhargav Kurma, Silpa Vadakkeeveetil Sreelatha, Manasi Patwardhan, and Shirish Karande. 2020. Understanding Advertisements with BERT. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7542–7547, Online. Association for Computational Linguistics.
Cite (Informal):
Understanding Advertisements with BERT (Kalra et al., ACL 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.acl-main.674.pdf
Video:
 http://slideslive.com/38929293