Out-of-domain Detection based on Generative Adversarial Network

The main goal of this paper is to develop out-of-domain (OOD) detection for dialog systems. We propose to use only in-domain (IND) sentences to build a generative adversarial network (GAN) of which the discriminator generates low scores for OOD sentences. To improve basic GANs, we apply feature matching loss in the discriminator, use domain-category analysis as an additional task in the discriminator, and remove the biases in the generator. Thereby, we reduce the huge effort of collecting OOD sentences for training OOD detection. For evaluation, we experimented OOD detection on a multi-domain dialog system. The experimental results showed the proposed method was most accurate compared to the existing methods.


Introduction
Multi-domain dialog systems (Hakkani-Tur et al., 2016;Jiang et al., 2014;Lee et al., 2013;Ryu et al., 2015;Seon et al., 2014) should detect whether an input request is out-of-domain (OOD) because users do not know the exact coverages of those systems. One important problem of building OOD detection is the huge effort required to collect OOD sentences. This paper focuses on developing an accurate OOD detection method that requires only in-domain (IND) sentences for training, so this paper can reduce the effort of collecting OOD sentences.
For OOD detection, sentences would be represented in a continuous vector space in which IND cases are distinguished from OOD cases. Therefore, we use the existing sentence embedding method for OOD detection (Ryu et al., 2017). The authors train a recurrent neural network (RNN) for domain-category analysis task, in which one domain-category is assigned to an input sentence. Due to the similarity between OOD detection and domain-category analysis, the extracted features (i.e., representation) of the RNN contain information about domain-category. In addition, the word representations are pre-trained from a large unlabelled corpus, so the sentence embedding method has the advantage in understanding rare or unknown words that are likely to appear in OOD sentences.
Afterwards, we use the learned representations of IND sentences to train one-class classifiers that distinguish IND sentences from OOD sentences. We propose to use a generative adversarial network (GAN) (Goodfellow et al., 2014) that consists of a generator G and a discriminator D. We train D that distinguishes the IND sentences from the fake sentences generated by G, so we expect D to reject OOD sentences. We apply three modifications to improve basic GANs. To the best of our knowledge, this is the first study that uses GANs to solve OOD detection. Lane et al. (2007) proposed an in-domain verification method. The authors first build a basic binary classifier for each domain, and then build a meta classifier that takes the scores by the basic binary classifiers as input. However, in our experiment, many OOD sentences were misclassified into IND because OOD sentences were not in the negative examples of the classifiers. Therefore, the confidence scores of the basic binary classifiers are not sufficiently reliable evidences of OOD. Also, understanding rare or unknown words remains a problem because bag-of-words model is used. Ryu et al. (2017) proposed an autoencoderbased method. The authors use neural sentence embedding that has the advantage in representing rare or unknown words. Based on those distributed sentence representations, an autoencoder is trained on IND sentences. The autencoder will have low reconstruction errors for IND sentences, so an input sentence can be classified into either IND or OOD. However, the autoencoder-based method has a limitation in expandability. When the weights are initialized carefully and regularization techniques are applied, the trained autoencoder reconstructs any input accurately. This result means that the reconstruction errors by the ideal autoencoder are not reliable evidence of OOD, so in OOD detection, autoencoders have little room for improvement.

Methods
As we discussed in Section 1, we use the sentence embedding to represent sentences in an 300dimensional continuous vector space. We propose to use a GAN to OOD detection; a GAN consists of two adversarial components: generator G and discriminator D. G generates artificial data to de-ceive D. D distinguishes real data from the artificial data generated by G. GAN is an unsupervised algorithm because learning G and D does not require labels. Standard GANs are trained based on the objective function V (D, G) as We propose to use GANs to obtain a one-class classifier for OOD detection. When we train G to generate sentences similar to IND sentences and D to classify real IND sentences and fake sentences generated by G, we expect D to reject OOD sentences. Therefore, we use the low confidence score by D about an input sentence as the evidence that the sentence is OOD.
Let p z (z) be a continuous uniform distribution (−1, 1). We define G that generates fake data G(z) ∈ R m from input noise z ∼ p z (z), f that extracts features from either real data x of (x, y) ∼ p data (x, y) or G(z), and D that measures the probability of either f(x) or f(G(z)) from the real data as ( where W g , W f , and W d are weight matrices, b f and b d are bias vectors. So we define two objective functions Because G(z) continuously changes during the training, f of traditional GAN also changes. Thus we define C ∈ R |D| that computes domaincategory of f(x) as where W c is a weight matrix and b c is a bias vector. We expect f trained by the losses of both D and C to be more stable than f trained by the loss of only D, so we define an objective function where H(p, q) is the categorical cross entropy and y is the true domain-category. In addition, GAN suffers from a mode collapse problem in which G generates samples with a low variance. To solve the problem, we remove the biases in the generator because the G was trained to use the biases mainly instead of the weights to generate data.
Second, we apply feature matching (Salimans et al., 2016). The authors say "Instead of directly maximizing the output of the discriminator, the new objective requires the generator to generate data that matches [sic] the statistics of the real data, where we use the discriminator only to specify the statistics that we think are worth matching". So G is trained to generate high variance sentence G(z) by additional objective function Based on our design of GAN, we train D, C, f, and G as Algorithm 1. To implement our GAN Algorithm 1 Training process of GAN for OOD detection.
for number of training iterations do Sample real data (x, y) ∼ p data (x, y). Sample noise z ∼ p z (z).
Update D, C, and f based on L D + L C . Sample noise z ∼ p z (z).
Update G based on L G + L f . end for for one-class classification, we use the Tensorflow library (Abadi et al., 2015). We train our models by using Adam (Kingma and Ba, 2015) optimizer with a mini-batch size of 256 and an initial learning rate of 0.01 that is decreased linearly during 500 epochs. All weights are initialized from a zero-centered Normal distribution with standard deviation 1.0.

Data Set
We experimented on a data set of 6,268 Korean sentences. We collected 706 OOD sentences about three domains: hotel, message, and smalltalk; and 5,562 IND sentences about fourteen domains: airplane, alarm, bus, call, car navigation, diet talk, exchange, general, schedule, songfinder, time, train, and TV control. We used eighty percent of the IND sentences to train the models; we used the remaining IND sentences and all OOD sentences for testing.

Evaluation Metrics
We use equal error rate (EER) to represent the accuracy of OOD detection (Lane et al., 2007). EER is the error rate at which false acceptance rate We performed each experiment 20 times, and recorded the average EER of OOD detection.

Compared Methods
We have three variations of vanilla GAN: to remove the biases, to add domain-category analy- We compare our method to three one-class classifiers. (1) Local outlier factor (Breunig et al., 2000) compares the local density of a point to the local densities of its neighbors, and considers the point that has lower density than their neighbors as an outlier. The local density of a point is defined by the distance to its nearest neighbors.
(2) One-class support vector machines (One-class SVMs) (Schölkopf et al., 2001) that treats the origin as a negative example to learn a decision function. (3) Autoencoder is explained in Section 2.

Results
In the experiments (Table 1), the best EER (8.96%) was obtained by the GAN in which all three of our variations are applied (p < 0.05). This result means that (1) removing the biases and using the feature matching prevented the generator from mode collapse problem and (2) using domain-category analysis as an auxiliary task stabilized the training of feature extractor. Compared to the other one-class classification methods including the autoencoder, the proposed GAN was most accurate (p < 0.05), so we can say that the discriminator scores of the GAN are reliable evidence for OOD detection (Table 2).

Conclusion
In this paper, we aimed at building OOD detection without OOD sentences for training. We proposed to use the discriminator of a GAN, which is trained on only IND sentences. The proposed method outperformed the existing methods in our data set. To train the GAN, we used the distributed sentence representations computed from the pretrained sentence embeddings instead of symbolic sentences. However, we think the limitation of the pre-trained sentence embeddings can be overcome by building a GAN that generates symbolic sentences and discriminates them.