Adding the Third Dimension to Spatial Relation Detection in 2D Images

Brandon Birmingham, Adrian Muscat, Anja Belz


Abstract
Detection of spatial relations between objects in images is currently a popular subject in image description research. A range of different language and geometric object features have been used in this context, but methods have not so far used explicit information about the third dimension (depth), except when manually added to annotations. The lack of such information hampers detection of spatial relations that are inherently 3D. In this paper, we use a fully automatic method for creating a depth map of an image and derive several different object-level depth features from it which we add to an existing feature set to test the effect on spatial relation detection. We show that performance increases are obtained from adding depth features in all scenarios tested.
Anthology ID:
W18-6517
Volume:
Proceedings of the 11th International Conference on Natural Language Generation
Month:
November
Year:
2018
Address:
Tilburg University, The Netherlands
Editors:
Emiel Krahmer, Albert Gatt, Martijn Goudbeek
Venue:
INLG
SIG:
SIGGEN
Publisher:
Association for Computational Linguistics
Note:
Pages:
146–151
Language:
URL:
https://aclanthology.org/W18-6517
DOI:
10.18653/v1/W18-6517
Bibkey:
Cite (ACL):
Brandon Birmingham, Adrian Muscat, and Anja Belz. 2018. Adding the Third Dimension to Spatial Relation Detection in 2D Images. In Proceedings of the 11th International Conference on Natural Language Generation, pages 146–151, Tilburg University, The Netherlands. Association for Computational Linguistics.
Cite (Informal):
Adding the Third Dimension to Spatial Relation Detection in 2D Images (Birmingham et al., INLG 2018)
Copy Citation:
PDF:
https://aclanthology.org/W18-6517.pdf