Learning Transferable Feature Representations Using Neural Networks

Himanshu Sharad Bhatt, Shourya Roy, Arun Rajkumar, Sriranjani Ramakrishnan


Abstract
Learning representations such that the source and target distributions appear as similar as possible has benefited transfer learning tasks across several applications. Generally it requires labeled data from the source and only unlabeled data from the target to learn such representations. While these representations act like a bridge to transfer knowledge learned in the source to the target; they may lead to negative transfer when the source specific characteristics detract their ability to represent the target data. We present a novel neural network architecture to simultaneously learn a two-part representation which is based on the principle of segregating source specific representation from the common representation. The first part captures the source specific characteristics while the second part captures the truly common representation. Our architecture optimizes an objective function which acts adversarial for the source specific part if it contributes towards the cross-domain learning. We empirically show that two parts of the representation, in different arrangements, outperforms existing learning algorithms on the source learning as well as cross-domain tasks on multiple datasets.
Anthology ID:
P19-1404
Volume:
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2019
Address:
Florence, Italy
Editors:
Anna Korhonen, David Traum, Lluís Màrquez
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
4124–4134
Language:
URL:
https://aclanthology.org/P19-1404
DOI:
10.18653/v1/P19-1404
Bibkey:
Cite (ACL):
Himanshu Sharad Bhatt, Shourya Roy, Arun Rajkumar, and Sriranjani Ramakrishnan. 2019. Learning Transferable Feature Representations Using Neural Networks. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4124–4134, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Learning Transferable Feature Representations Using Neural Networks (Bhatt et al., ACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/P19-1404.pdf