Deep Neural Model Inspection and Comparison via Functional Neuron Pathways

James Fiacco, Samridhi Choudhary, Carolyn Rose


Abstract
We introduce a general method for the interpretation and comparison of neural models. The method is used to factor a complex neural model into its functional components, which are comprised of sets of co-firing neurons that cut across layers of the network architecture, and which we call neural pathways. The function of these pathways can be understood by identifying correlated task level and linguistic heuristics in such a way that this knowledge acts as a lens for approximating what the network has learned to apply to its intended task. As a case study for investigating the utility of these pathways, we present an examination of pathways identified in models trained for two standard tasks, namely Named Entity Recognition and Recognizing Textual Entailment.
Anthology ID:
P19-1575
Volume:
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2019
Address:
Florence, Italy
Editors:
Anna Korhonen, David Traum, Lluís Màrquez
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5754–5764
Language:
URL:
https://aclanthology.org/P19-1575
DOI:
10.18653/v1/P19-1575
Bibkey:
Cite (ACL):
James Fiacco, Samridhi Choudhary, and Carolyn Rose. 2019. Deep Neural Model Inspection and Comparison via Functional Neuron Pathways. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5754–5764, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Deep Neural Model Inspection and Comparison via Functional Neuron Pathways (Fiacco et al., ACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/P19-1575.pdf
Video:
 https://aclanthology.org/P19-1575.mp4
Data
CoNLL 2003MultiNLI