Neural Natural Language Inference Models Enhanced with External Knowledge

Modeling natural language inference is a very challenging task. With the availability of large annotated data, it has recently become feasible to train complex models such as neural-network-based inference models, which have shown to achieve the state-of-the-art performance. Although there exist relatively large annotated data, can machines learn all knowledge needed to perform natural language inference (NLI) from these data? If not, how can neural-network-based NLI models benefit from external knowledge and how to build NLI models to leverage it? In this paper, we enrich the state-of-the-art neural natural language inference models with external knowledge. We demonstrate that the proposed models improve neural NLI models to achieve the state-of-the-art performance on the SNLI and MultiNLI datasets.


Introduction
Reasoning and inference are central to both human and artificial intelligence. Natural language inference (NLI), also known as recognizing textual entailment (RTE), is an important NLP problem concerned with determining inferential relationship (e.g., entailment, contradiction, or neutral) between a premise p and a hypothesis h. In general, modeling informal inference in language is a very challenging and basic problem towards achieving true natural language understanding.
In the last several years, larger annotated datasets were made available, e.g., the SNLI (Bowman et al., 2015) and MultiNLI datasets (Williams et al., 2017), which made it feasible to train rather complicated neuralnetwork-based models that fit a large set of parameters to better model NLI. Such models have shown to achieve the state-of-the-art performance (Bowman et al., 2015(Bowman et al., , 2016Yu and Munkhdalai, 2017b;Parikh et al., 2016;Sha et al., 2016;Chen et al., 2017a,b;Tay et al., 2018). While neural networks have been shown to be very effective in modeling NLI with large training data, they have often focused on end-to-end training by assuming that all inference knowledge is learnable from the provided training data. In this paper, we relax this assumption and explore whether external knowledge can further help NLI. Consider an example: • p: A lady standing in a wheat field.
• h: A person standing in a corn field.
In this simplified example, when computers are asked to predict the relation between these two sentences and if training data do not provide the knowledge of relationship between "wheat" and "corn" (e.g., if one of the two words does not appear in the training data or they are not paired in any premise-hypothesis pairs), it will be hard for computers to correctly recognize that the premise contradicts the hypothesis.
In general, although in many tasks learning tabula rasa achieved state-of-the-art performance, we believe complicated NLP problems such as NLI could benefit from leveraging knowledge accumulated by humans, particularly in a foreseeable future when machines are unable to learn it by themselves.
In this paper we enrich neural-network-based NLI models with external knowledge in coattention, local inference collection, and inference composition components. We show the proposed model improves the state-of-the-art NLI models to achieve better performances on the SNLI and MultiNLI datasets. The advantage of using external knowledge is more significant when the size of training data is restricted, suggesting that if more knowledge can be obtained, it may bring more benefit. In addition to attaining the state-of-theart performance, we are also interested in understanding how external knowledge contributes to the major components of typical neural-networkbased NLI models.

Related Work
Early research on natural language inference and recognizing textual entailment has been performed on relatively small datasets (refer to MacCartney (2009) for a good literature survey), which includes a large bulk of contributions made under the name of RTE, such as (Dagan et al., 2005;Iftene and Balahur-Dobrescu, 2007), among many others.
More recently the availability of much larger annotated data, e.g., SNLI (Bowman et al., 2015) and MultiNLI (Williams et al., 2017), has made it possible to train more complex models. These models mainly fall into two types of approaches: sentence-encoding-based models and models using also inter-sentence attention. Sentence-encoding-based models use Siamese architecture (Bromley et al., 1993). The parametertied neural networks are applied to encode both the premise and the hypothesis. Then a neural network classifier is applied to decide relationship between the two sentences. Different neural networks have been utilized for sentence encoding, such as LSTM (Bowman et al., 2015), GRU (Vendrov et al., 2015), CNN (Mou et al., 2016), BiL-STM and its variants (Liu et al., 2016c;Lin et al., 2017;Chen et al., 2017b;Nie and Bansal, 2017), self-attention network (Shen et al., 2017(Shen et al., , 2018, and more complicated neural networks (Bowman et al., 2016;Yu and Munkhdalai, 2017a,b;Choi et al., 2017). Sentence-encoding-based models transform sentences into fixed-length vector representations, which may help a wide range of tasks (Conneau et al., 2017).
The second set of models use inter-sentence attention (Rocktäschel et al., 2015;Wang and Jiang, 2016;Cheng et al., 2016;Parikh et al., 2016;. Among them, Rocktäschel et al. (2015) were among the first to propose neural attention-based models for NLI.  proposed an enhanced sequential inference model (ESIM), which is one of the best models so far and is used as one of our baselines in this paper.
In this paper we enrich neural-network-based NLI models with external knowledge. Unlike early work on NLI (Jijkoun and de Rijke, 2005;MacCartney et al., 2008;MacCartney, 2009) that explores external knowledge in conventional NLI models on relatively small NLI datasets, we aim to merge the advantage of powerful modeling ability of neural networks with extra external inference knowledge. We show that the proposed model improves the state-of-the-art neural NLI models to achieve better performances on the SNLI and MultiNLI datasets. The advantage of using external knowledge is more significant when the size of training data is restricted, suggesting that if more knowledge can be obtained, it may have more benefit. In addition to attaining the state-of-the-art performance, we are also interested in understanding how external knowledge affect major components of neural-network-based NLI models.

Neural-Network-Based NLI Models with External Knowledge
In this section we propose neural-network-based NLI models to incorporate external inference knowledge, which, as we will show later in Section 5, achieve the state-of-the-art performance. In addition to attaining the leading performance we are also interested in investigating the effects of external knowledge on major components of neural-network-based NLI modeling. Figure 1 shows a high-level general view of the proposed framework. While specific NLI systems vary in their implementation, typical state-of-theart NLI models contain the main components (or equivalents) of representing premise and hypothesis sentences, collecting local (e.g., lexical) inference information, and aggregating and composing local information to make the global decision at the sentence level. We incorporate and investigate external knowledge accordingly in these major NLI components: computing co-attention, collecting local inference information, and composing inference to make final decision.

External Knowledge
As discussed above, although there exist relatively large annotated data for NLI, can machines learn all inference knowledge needed to perform NLI from the data? If not, how can neural networkbased NLI models benefit from external knowledge and how to build NLI models to leverage it?
We study the incorporation of external, inference-related knowledge in major components of neural networks for natural language inference. For example, intuitively knowledge about synonymy, antonymy, hypernymy and hyponymy between given words may help model soft-alignment between premises and hypotheses; knowledge about hypernymy and hyponymy may help capture entailment; knowledge about antonymy and co-hyponyms (words sharing the same hypernym) may benefit the modeling of contradiction.
In this section, we discuss the incorporation of basic, lexical-level semantic knowledge into neural NLI components. Specifically, we consider external lexical-level inference knowledge between word w i and w j , which is represented as a vector r ij and is incorporated into three specific components shown in Figure 1. We will discuss the details of how r ij is constructed later in the experiment setup section (Section 4) but instead focus on the proposed model in this section. Note that while we study lexical-level inference knowledge in the paper, if inference knowledge about larger pieces of text pairs (e.g., inference relations between phrases) are available, the proposed model can be easily extended to handle that. In this paper, we instead let the NLI models to compose lexicallevel knowledge to obtain inference relations between larger pieces of texts.

Encoding Premise and Hypothesis
Same as much previous work (Chen et al., 2017a,b), we encode the premise and the hypothesis with bidirectional LSTMs (BiLSTMs). The premise is represented as a = (a 1 , . . . , a m ) and the hypothesis is b = (b 1 , . . . , b n ), where m and n are the lengths of the sentences. Then a and b are embedded into d e -dimensional vectors where |V | is the vocabulary size and E can be initialized with the pre-trained word embedding. To represent words in its context, the premise and the hypothesis are fed into BiLSTM encoders (Hochreiter and Schmidhuber, 1997) to obtain context-dependent hidden states a s and b s : where i and j indicate the i-th word in the premise and the j-th word in the hypothesis, respectively.

Knowledge-Enriched Co-Attention
As discussed above, soft-alignment of word pairs between the premise and the hypothesis may benefit from knowledge-enriched co-attention mechanism. Given the relation features r ij ∈ R dr between the premise's i-th word and the hypothesis's j-th word derived from the external knowledge, the co-attention is calculated as: The function F can be any non-linear or linear functions. In this paper, we use F (r ij ) = λ1(r ij ), where λ is a hyper-parameter tuned on the development set and 1 is the indication function as follows: Intuitively, word pairs with semantic relationship, e.g., synonymy, antonymy, hypernymy, hyponymy and co-hyponyms, are probably aligned together. We will discuss how we construct external knowledge later in Section 4. We have also tried a twolayer MLP as a universal function approximator in function F to learn the underlying combination function but did not observe further improvement over the best performance we obtained on the development datasets. Soft-alignment is determined by the coattention matrix e ∈ R m×n computed in Equation (3), which is used to obtain the local relevance between the premise and the hypothesis. For the hidden state of the i-th word in the premise, i.e., a s i (already encoding the word itself and its context), the relevant semantics in the hypothesis is identified into a context vector a c i using e ij , more specifically with Equation (5).
where α ∈ R m×n and β ∈ R m×n are the normalized attention weight matrices with respect to the 2-axis and 1-axis. The same calculation is performed for each word in the hypothesis, i.e., b s j , with Equation (6) to obtain the context vector b c j .

Local Inference Collection with External Knowledge
By way of comparing the inference-related semantic relation between a s i (individual word representation in premise) and a c i (context representation from hypothesis which is align to word a s i ), we can model local inference (i.e., word-level inference) between aligned word pairs. Intuitively, for example, knowledge about hypernymy or hyponymy may help model entailment and knowledge about antonymy and co-hyponyms may help model contradiction. Through comparing a s i and a c i , in addition to their relation from external knowledge, we can obtain word-level inference information for each word. The same calculation is performed for b s j and b c j . Thus, we collect knowledge-enriched local inference information: where a heuristic matching trick with difference and element-wise product is used (Mou et al., 2016;. The last terms in Equation (7)(8) are used to obtain word-level inference information from external knowledge. Take Equation (7) as example, r ij is the relation feature between the i-th word in the premise and the j-th word in the hypothesis, but we care more about semantic relation between aligned word pairs between the premise and the hypothesis. Thus, we use a soft-aligned version through the soft-alignment weight α ij . For the i-th word in the premise, the last term in Equation (7) is a word-level inference information based on external knowledge between the i-th word and the aligned word. The same calculation for hypothesis is performed in Equation (8). G is a nonlinear mapping function to reduce dimensionality. Specifically, we use a 1-layer feed-forward neural network with the ReLU activation function with a shortcut connection, i.e., concatenate the hidden states after ReLU with the input n j=1 α ij r ij (or m i=1 β ij r ji ) as the output a m i (or b m j ).

Knowledge-Enhanced Inference Composition
In this component, we introduce knowledgeenriched inference composition. To determine the overall inference relationship between the premise and the hypothesis, we need to explore a composition layer to compose the local inference vectors (a m and b m ) collected above: Here, we also use BiLSTMs as building blocks for the composition layer, but the responsibility of BiLSTMs in the inference composition layer is completely different from that in the input encoding layer. The BiLSTMs here read local inference vectors (a m and b m ) and learn to judge the types of local inference relationship and distinguish crucial local inference vectors for overall sentence-level inference relationship. Intuitively, the final prediction is likely to depend on word pairs appearing in external knowledge that have some semantic relation. Our inference model converts the output hidden vectors of BiLSTMs to the fixed-length vector with pooling operations and puts it into the final classifier to determine the overall inference class. Particularly, in addition to using mean pooling and max pooling similarly to ESIM , we propose to use weighted pooling based on external knowledge to obtain a fixed-length vector as in Equation (11)(12).
In our experiments, we regard the function H as a 1-layer feed-forward neural network with ReLU activation function. We concatenate all pooling vectors, i.e., mean, max, and weighted pooling, into the fixed-length vector and then put the vector into the final multilayer perceptron (MLP) classifier. The MLP has one hidden layer with tanh activation and softmax output layer in our experiments. The entire model is trained end-to-end, through minimizing the cross-entropy loss.
4 Experiment Set-Up

Representation of External Knowledge
Lexical Semantic Relations As described in Section 3.1, to incorporate external knowledge (as a knowledge vector r ij ) to the state-of-theart neural network-based NLI models, we first explore semantic relations in WordNet (Miller, 1995), motivated by MacCartney (2009). Specifically, the relations of lexical pairs are derived as described in (1)-(4) below. Instead of using Jiang-Conrath WordNet distance metric (Jiang and Conrath, 1997), which does not improve the performance of our models on the development sets, we add a new feature, i.e., co-hyponyms, which consistently benefit our models.
(2) Antonymy: It takes the value 1 if the words in the pair are antonyms in WordNet, and 0 otherwise. For example, [wet, dry] = 1.
(3) Hypernymy: It takes the value 1 − n/8 if one word is a (direct or indirect) hypernym of the other word in WordNet, where n is the number of edges between the two words in hierarchies, and 0 otherwise. Note that we ignore pairs in the hierarchy which have more than 8 edges in between. (5) Co-hyponyms: It takes the value 1 if the two words have the same hypernym but they do not belong to the same synset, and 0 otherwise. For example, [dog, wolf] = 1.
As discussed above, we expect features like synonymy, antonymy, hypernymy, hyponymy and cohyponyms would help model co-attention alignment between the premise and the hypothesis. Knowledge of hypernymy and hyponymy may help capture entailment; knowledge of antonymy and co-hyponyms may help model contradiction. Their final contributions will be learned in end-to-end model training. We regard the vector r ∈ R dr as the relation feature derived from external knowledge, where d r is 5 here. In addition, Table 1 reports some key statistics of these features.  In addition to the above relations, we also use more relation features in WordNet, including instance, instance of, same instance, entailment, member meronym, member holonym, substance meronym, substance holonym, part meronym, part holonym, summing up to 15 features, but these additional features do not bring further improvement on the development dataset, as also discussed in Section 5.

Relation Embeddings
In the most recent years graph embedding has been widely employed to learn representation for vertexes and their relations in a graph. In our work here, we also capture the relation between any two words in WordNet through relation embedding. Specifically, we employed TransE (Bordes et al., 2013), a widely used graph embedding methods, to capture relation embedding between any two words. We used two typical approaches to obtaining the relation embedding. The first directly uses 18 relation embeddings pretrained on the WN18 dataset (Bordes et al., 2013). Specifically, if a word pair has a certain type relation, we take the corresponding relation embedding. Sometimes, if a word pair has multiple relations among the 18 types; we take an average of the relation embedding. The second approach uses TransE's word embedding (trained on WordNet) to obtain relation embedding, through the objective function used in TransE, i.e., l ≈ t − h, where l indicates relation embedding, t indicates tail entity embedding, and h indicates head entity embedding.
Note that in addition to relation embedding trained on WordNet, other relational embedding resources exist; e.g., that trained on Freebase (WikiData) (Bollacker et al., 2007), but such knowledge resources are mainly about facts (e.g., relationship between Bill Gates and Microsoft) and are less for commonsense knowledge used in general natural language inference (e.g., the color yellow potentially contradicts red).

NLI Datasets
In our experiments, we use Stanford Natural Language Inference (SNLI) dataset (Bowman et al., 2015) and Multi-Genre Natural Language Inference (MultiNLI) (Williams et al., 2017) dataset, which focus on three basic relations between a premise and a potential hypothesis: the premise entails the hypothesis (entailment), they contradict each other (contradiction), or they are not related (neutral). We use the same data split as in previous work (Bowman et al., 2015;Williams et al., 2017) and classification accuracy as the evaluation metric. In addition, we test our models (trained on the SNLI training set) on a new test set (Glockner et al., 2018), which assesses the lexical inference abilities of NLI systems and consists of 8,193 samples. WordNet 3.0 (Miller, 1995) is used to extract semantic relation features between words. The words are lemmatized using Stanford CoreNLP 3.7.0 . The premise and the hypothesis sentences fed into the input encoding layer are tokenized.

Training Details
For duplicability, we release our code 1 . All our models were strictly selected on the development set of the SNLI data and the in-domain development set of MultiNLI and were then tested on the corresponding test set. The main training details are as follows: the dimension of the hidden states of LSTMs and word embeddings are 300. The word embeddings are initialized by 300D GloVe 840B (Pennington et al., 2014), and out-of-vocabulary words among them are initialized randomly. All word embeddings are updated during training. Adam (Kingma and Ba, 2014) is used for optimization with an initial learning rate of 0.0004. The mini-batch size is set to 32. Note that the above hyperparameter settings are same as those used in the baseline ESIM  model. ESIM is a strong NLI baseline framework with the source code made available at https://github.com/lukecq1231/nli (the ESIM core code has also been adapted to summarization  and questionanswering tasks (Zhang et al., 2017a)).
The trade-off λ for calculating co-attention in Equation (3) is selected in [0.1, 0.2, 0.5, 1, 2, 5, 10, 20, 50] based on the development set. When training TransE for WordNet, relations are represented with vectors of 20 dimension. Table 2 shows the results of state-of-the-art models on the SNLI dataset. Among them, ESIM  is one of the previous state-of-the-art systems with an 88.0% test-set accuracy. The proposed model, namely Knowledge-based Inference Model (KIM), which enriches ESIM with external knowledge, obtains an accuracy of 88.6%, the best single-model performance reported on the SNLI dataset. The difference between ESIM and KIM is statistically significant under the one-tailed paired t-test at the 99% significance level. Note that the KIM model reported here uses five semantic relations described in Section 4. In addition to that, we also use 15 semantic relation features, which does not bring additional gains in performance. These results highlight the effectiveness of the five semantic relations described in Section 4. To further investigate external knowledge, we add TransE relation embedding, and again no further improvement is observed on both the development and test sets when TransE relation embedding is used (concatenated) with the semantic relation vectors. We consider this is due to the fact that TransE embedding is not specifically sensitive to inference information; e.g., it does not model co-hyponyms features, and its potential benefit has already been covered by the semantic relation features used. Table 3 shows the performance of models on the MultiNLI dataset. The baseline ESIM achieves 76.8% and 75.8% on in-domain and cross-domain test set, respectively. If we extend the ESIM with external knowledge, we achieve significant gains to 77.2% and 76.4% respectively. Again, the gains are consistent on SNLI and MultiNLI, and we expect they would be orthogonal to other factors when external knowledge is added into other stateof-the-art models.

Ablation Results
Figure 2 displays the ablation analysis of different components when using the external knowledge. To compare the effects of external knowledge under different training data scales, we ran-

Model In Cross
CBOW (Williams et al., 2017) 64.8 64.5 BiLSTM (Williams et al., 2017) 66.9 66.9 DiSAN (Shen et al., 2017) 71.0 71.4 Gated BiLSTM (Chen et al., 2017b) 73.5 73.6 SS BiLSTM (Nie and Bansal, 2017) 74.6 73.6 DIIN * (Gong et al., 2017) 77.8 78.8 CAFE (Tay et al., 2018) 78.7 77.9 ESIM    domly sample different ratios of the entire training set, i.e., 0.8%, 4%, 20% and 100%. "A" indicates adding external knowledge in calculating the coattention matrix as in Equation (3), "I" indicates adding external knowledge in collecting local inference information as in Equation (7)(8), and "C" indicates adding external knowledge in composing inference as in Equation (11)(12). When we only have restricted training data, i.e., 0.8% training set (about 4,000 samples), the baseline ESIM has a poor accuracy of 62.4%. When we only add external knowledge in calculating co-attention ("A"), the accuracy increases to 66.6% (+ absolute 4.2%). When we only utilize external knowledge in collecting local inference information ("I"), the accuracy has a significant gain, to 70.3% (+ absolute 7.9%). When we only add external knowledge in inference composition ("C"), the accuracy gets a smaller gain to 63.4% (+ absolute 1.0%). The comparison indicates that "I" plays the most important role among the three components in using external knowledge. Moreover, when we com-pose the three components ("A,I,C"), we obtain the best result of 72.6% (+ absolute 10.2%). When we use more training data, i.e., 4%, 20%, 100% of the training set, only "I" achieves a significant gain, but "A" or "C" does not bring any significant improvement. The results indicate that external semantic knowledge only helps co-attention and composition when limited training data is limited, but always helps in collecting local inference information. Meanwhile, for less training data, λ is usually set to a larger value. For example, the optimal λ on the development set is 20 for 0.8% training set, 2 for the 4% training set, 1 for the 20% training set and 0.2 for the 100% training set. Figure 3 displays the results of using different ratios of external knowledge (randomly keep different percentages of whole lexical semantic relations) under different sizes of training data. Note that here we only use external knowledge in collecting local inference information as it always works well for different scale of the training set. Better accuracies are achieved when using more external knowledge. Especially under the condition of restricted training data (0.8%), the model obtains a large gain when using more than half of external knowledge. Figure 2: Accuracies of models of incorporating external knowledge into different NLI components, under different sizes of training data (0.8%, 4%, 20%, and the entire training data). (Glockner et al., 2018) Test Set

Analysis on the
In addition, Table 4 shows the results on a newly published test set (Glockner et al., 2018). Compared with the performance on the SNLI test   (Glockner et al., 2018).
set, the performance of the three baseline models dropped substantially on the (Glockner et al., 2018) test set, with the differences ranging from 22.3% to 32.8% in accuracy. Instead, the proposed KIM achieves 83.5% on this test set (with only a 5.1% drop in performance), which demonstrates its better ability of utilizing lexical level inference and hence better generalizability. Figure 5 displays the accuracy of ESIM and KIM in each replacement-word category of the (Glockner et al., 2018) test set. KIM outperforms ESIM in 13 out of 14 categories, and only performs worse on synonyms.

Analysis by Inference Categories
We perform more analysis (Table 6) using the supplementary annotations provided by the MultiNLI dataset (Williams et al., 2017), which have 495 samples (about 1/20 of the entire development set) for both in-domain and out-domain set. We compare against the model outputs of the ESIM model across 13 categories of inference. Table 6 reports the results. We can see that KIM outperforms ESIM on overall accuracies on both in-domain and   cross-domain subset of development set. KIM outperforms or equals ESIM in 10 out of 13 categories on the cross-domain setting, while only 7 out of 13 categories on in-domain setting. It indicates that external knowledge helps more in crossdomain setting. Especially, for antonym category in cross-domain set, KIM outperform ESIM significantly (+ absolute 5.0%) as expected, because antonym feature captured by external knowledge would help unseen cross-domain samples.  ple, the premise is "An African person standing in a wheat field" and the hypothesis "A person standing in a corn field". As the KIM model knows that "wheat" and "corn" are both a kind of cereal, i.e, the co-hyponyms relationship in our relation features, KIM therefore predicts the premise contradicts the hypothesis. However, the baseline ESIM cannot learn the relationship between "wheat" and "corn" effectively due to lack of enough samples in the training sets. With the help of external knowledge, i.e., "wheat" and "corn" having the same hypernym "cereal", KIM predicts contradiction correctly.

Conclusions
Our neural-network-based model for natural language inference with external knowledge, namely KIM, achieves the state-of-the-art accuracies. The model is equipped with external knowledge in its main components, specifically, in calculating coattention, collecting local inference, and composing inference. We provide detailed analyses on our model and results. The proposed model of infusing neural networks with external knowledge may also help shed some light on tasks other than NLI.