Pre-train and Plug-in: Flexible Conditional Text Generation with Variational Auto-Encoders

Conditional Text Generation has drawn much attention as a topic of Natural Language Generation (NLG) which provides the possibility for humans to control the properties of generated contents. Current conditional generation models cannot handle emerging conditions due to their joint end-to-end learning fashion. When a new condition added, these techniques require full retraining. In this paper, we present a new framework named Pre-train and Plug-in Variational Auto-Encoder (PPVAE) towards flexible conditional text generation. PPVAE decouples the text generation module from the condition representation module to allow “one-to-many” conditional generation. When a fresh condition emerges, only a lightweight network needs to be trained and works as a plug-in for PPVAE, which is efficient and desirable for real-world applications. Extensive experiments demonstrate the superiority of PPVAE against the existing alternatives with better conditionality and diversity but less training effort.


Introduction
Currently, neural generation techniques have powered many inspiring applications, e.g., poem generation (Yang et al., 2018), neural machine translation (NMT) (Bahdanau et al., 2015) and chatbot (Zhao et al., 2017).Conditional (also known as controllable) text generation is an important task of text generation, aiming to generate realistic text that carries a specific attribute (e.g., positive or negative sentiment).A common solution is to encode the condition into a vector representation and then integrate it with the text generation process (Kingma et al., 2014;Hu et al., 2017;Mirza and Osindero, 2014).These existing neural models have achieved encouraging results.However, when a new condition is added (e.g., a new topic for categorical generation), they require a full retraining or finetuning.This process is both time-consuming and computationally inefficient (Houlsby et al., 2019).Both fine-tuning and retraining are not desirable in real-world applications since the delivery (e.g., transmitting updated weights through the Internet) and client-side re-deployment (e.g., distribute updated weights to users) of large-scale weights are often difficult.
Inspired by the recent success of Variational Auto-Encoder (VAE) (Kingma and Welling, 2014) based post-hoc conditional image generation strategy (Engel et al., 2018), we provide a new perspective for flexible conditional text generation.We propose Pre-train and Plug-in Variational Auto-Encoder (PPVAE), which decouples the text generation module from the condition representation module.PPVAE is a hierarchical framework composed of two VAEs: (1) PRETRAINVAE, which derives the global latent space of text with its encoder (pre-trained global encoder) and learns to generate text based on an easily-accessible large unlabeled dataset with its decoder (pre-trained global decoder); (2) PLUGINVAE, which is a lightweight neural network that learns to transform vectors from the conditional latent space to the global latent space, and vice versa.This mapping function can be easily learned with only a few conditional training samples.In this sense, once we transform a latent variable (also known as latent code) randomly sampled from the conditional space distribution to the global space, the pre-trained global decoder is directly adopted for generation.In other words, whenever a new condition emerges, we only need to train a PLUGINVAE and directly plug it into the framework.
Different from the existing end-to-end neural models (Mirza and Osindero, 2014;Sohn et al., 2015;Kingma et al., 2014), PPVAE focuses on the learning of pure transformation between the continuous latent spaces, instead of the tricky discrete text generation.Once trained, PRETRAINVAE is fixed for text representation and generation under all conditions.Our proposed framework decouples the conditional space learning from the text generation, endowing PPVAE with more flexibility when handling emerging conditions.Also, training only a small conditional network for latent space transformation is much more efficient than co-training with the text generation.Additionally, we can easily increase the capability of generation using a larger corpus or deeper neural networks for text encoding and decoding.Our main contributions can be summarized as follows: (1) We propose a novel framework, PPVAE, for conditional text generation, which allows a separate training for a new condition without retraining the whole network.(2) We conduct extensive experiments and analysis to verify the effectiveness of our proposed PPVAE.Our framework achieves state-of-the-art performance on conditionality in both automatic and human evaluations.

Related work
Boosted by the recent success of deep learning technology, Natural Language Generation (NLG) has recently become popular in the NLP community.Many great works have attempted to solve various subtasks like dialogue generation (Li et al., 2016), poetry generation (Yi et al., 2018) and story generation (Fan et al., 2018) and new techniques keep emerging (Bowman et al., 2016;Yu et al., 2017;Zhou et al., 2020).However, due to the blackbox nature of neural networks, the recent proposed generic models suffer the problem of lacking interpretability and controllability.
To handle this problem and support generating plausible text with a specified condition, conditional text generation (Kikuchi et al., 2016;Ficler and Goldberg, 2017;Hu et al., 2017) has recently attracted extensive attention.Current research in this direction mainly falls into two fashions: the supervised methods and semi-supervised methods.For supervised methods, Mirza and Osindero (2014); Sohn et al. (2015) first converted the condition information to one-hot vectors, then integrated them into a generator and a discriminator.To enhance the correlation between structured conditional code and generated samples, Chen et al. (2016) adopted an extra adversarial classifier to infer the structured code from generated samples.Wang and Wan (2018) used multiple generators for multiple conditions and a multi-class classifier to provide training signals for the learning of generators.
However, given only a limited number of conditional samples, semi-supervised methods are compulsory.To utilize the implicit conditional distribution behind the unlabeled text, Kingma et al. (2014) introduced a classifier into the VAE architecture.Hu et al. (2017) further involved two additional independent regularization terms in enhancing the disentanglement between structured code and unstructured code.Very recently, Keskar et al. (2019) used human-defined "control code" to pre-trained Language Model in an unsupervised manner.
Our work falls in the category of semisupervised learning yet differs from the existing works in the following ways: (1) Our model decouples the text generation module from the condition representation module which two are tightly fused as a single one in previous studies, enabling possible exploitation for pre-trained Language Models (e.g., GPT-2 (Radford et al., 2019)).( 2) Our model allows single-condition generation, which could inspire new applications like polite speech generator (Niu and Bansal, 2018) and data augmentation (Guo et al., 2018).(3) Our model can handle emerging conditions while achieving state-of-theart performance with fewer parameters and less training time.

Preliminaries
Variational Auto-Encoder (VAE).VAE (Kingma and Ba, 2015) is widely used in continuous generation (e.g., image generation).Bowman et al. (2016) introduced VAE to NLG to solve the "one-to-many" generation problem (i.e., generating multiple feasible samples for the same input).Given a latent variable z randomly sampled from a prior distribution, VAE comprises an encoder enc(x) = q φ (z|x) and a decoder dec(z) = p θ (x|z).The encoder aims to encode input data x into latent space Z ∈ R d .The decoder is used to reconstruct the original input x, given the corresponding z.Thus, the loss function of VAE is formulated as: where KL(•||•) is the Kullback-Leibler (KL) divergence, p(z) = N (0, 1) is the prior distribution.
The first term ensures that VAE can distill compact variable z in latent space for reconstruction.The second term pushes posterior distribution to be close to the prior distribution, securing the mutual information between original data and the latent space (Dupont, 2018).
Conditional Text Generation with VAE.Conditional text generation has drawn much attention recently.By controlling the properties of generated contents, we can apply the generative models to many real-world scenarios.We follow the problem setting in (Hu et al., 2017).Given a set of k conditions  et al., 2018).Specifically, we adopt WAE-GAN, a variant of WAE, which incorporates the merits of adversarial learning.During training, the encoder enc g (x) = q g (z g |x) encodes the text to the latent space and the decoder dec g (z g ) = p g (x|z g ) reconstruct the text with the latent variable z g .Thus, the loss function of PRETRAINVAE is formulated as: (2) where Q(z g ) = q g (z g |x)p(x) dx is the aggregated posterior distribution; p(z g ) is the prior normal distribution; D is the adversarial discriminator; λ is the coefficient hyper-parameter (λ > 0).PLUGINVAE.For each condition, we use a condition-specific PLUGINVAE to derive the conditional space.That is, PLUGINVAE is proposed to learn the transformation between the conditional and global latent space for each condition.Specifically, for each condition c i , we use a limited number of conditional samples y i and utilize the global encoder enc g to encode them into v y i .Note that normally, the encoded text samples under a single condition are not likely to densely clustered in the global text space Z g , since the learning process of Z g is condition-independent and the unlabeled corpus contains diverse text samples.PLUGINVAE for condition c i consists of an encoder enc c i (v y i ) = q c i (z c i |v y i ) and a decoder where d c is the space dimension.Thus, PLUGINVAE is capable of mapping the samples in the global latent space to and from a denser conditional latent space (i.e., d c < d g ).During training, the loss function of PLUGINVAE for a single condition is defined as: (3) where p(z c i ) is the prior normal distribution of the conditional latent space; z c i is the latent variable; To enhance the diversity of generated text, we introduce an extra constant term β to control the amount < l a t e x i t s h a 1 _ b a s e 6 4 = " 3 P D w N 7 9 6 q 0 f K a g 9 k 3 < l a t e x i t s h a 1 _ b a s e 6 4 = " a r N q 5 V u 7 S J y 8 6 < l a t e x i t s h a 1 _ b a s e 6 4 = " a r N q 5 V u 7 S J y 8 6 m G l P K h j T C j q W S x q j 9 8 e z U C T m 1 S p + E i b I l D Z m p v y f G N N Z 6 F A e 2 M 6 Z m o B e 9 q f i f 1 8 l M e O W P u U w z g 5 L N F 4 W Z I C Y h 0 7 9 J n y t k R o w s o U x x e y t h A 6 o o M z a d o g 3 B W 3 x 5 m T S r F e + 8 U r 2 7 K N e u 8 z g K c A w n c A Y e X E I N b q E O D W A Q w T O 8 w p s j n B f n 3 f m Y t 6 4 4 + c w R / I H z + Q M x I o 2 8 < / l a t e x i t > Z g < l a t e x i t s h a 1 _ b a s e 6 4 = " a r N q 5 V u 7 S J y 8 6 < l a t e x i t s h a 1 _ b a s e 6 4 = " a r N q 5 V u 7 S J y 8 6 of encoded information in VAE (Dupont, 2018;Chen et al., 2018;Kim and Mnih, 2018).By setting β to an appropriate value, PLUGINVAE could extract compact conditional information without sacrificing the fluency or accuracy.
Although we can already generate conditional text under a single condition by Equation 3, it is possible to even further improve the conditionality by introducing negative samples.We construct the negative samples y i from Y i and encode them: Thus, the loss function of PLUGINVAE with negative samples is defined as: where v y i is a batch of encoded samples under condition c i , and v y i is a batch of encoded negative samples; γ is a hyper-parameter balancing the positive and negative samples.For different tasks, the best setting for γ may vary.Intuitively, the larger the difference between the conditions is, the smaller γ should be.

Workflow
In this section, we provide the details of training and generation procedures.As illustrated in Figure 1, the workflow is composed of three steps.Pre-train once, infer everywhere.First, as shown in Figure 1(a), using the unlabeled corpus X, we pre-train PRETRAINVAE to learn the global latent space Z g by reconstruction with Equation 2. Once pre-trained, the weights of both enc g and dec g are fixed.As an unsupervised VAE model, PRETRAINVAE is capable of generating diverse but unconditional text.
Train it when you need it.Previous methods (Kingma et al., 2014;Hu et al., 2017) learn the joint conditional space by jointly considering all conditions.However, once the model is trained, it is not possible to add a new condition without a full retraining.Different from those approaches, PPVAE is totally flexible that allows adding new conditions.Shown in Figure 1(b), once a condition is added, we only need to train a PLUG-INVAE specifically for this condition with Equation 3 (or Equation 5, if provided with samples of other conditions).Since PLUGINVAE is textirrelevant and only learns to map between two latent spaces, the training number of parameters is only 0.34% (see Section 6.3) of fine-tuning PRE-TRAINVAE or retraining other models.Additionally, although we need to train k PLUGINVAE for k conditions, the total number of trained parameters is still much smaller than existing methods (unless k > 1/0.34% ≈ 294, which is impossible in actual applications).Plus, we can parallel the conditional training to speed up the process easily.
Plug it in and generate.Shown in Figure 1(c), once PLUGINVAE for the condition c i is trained, we can plug it into the PPVAE framework and generate text together with PRETRAINVAE.First, we randomly sample a latent variable z c i from the prior distribution p(z c i ) = N (0, 1).Then we use PLUGINVAE's decoder dec c i to map z c i to the global latent space Z g and obtain z c i : Since z c i ∈ Z g , we can directly use the global decoder dec g to generate text: where ŷi is the generated text under condition c i .
5 Experimental Settings

Datasets
Following the setting of (Hu et al., 2017) than 15 tokens), which is easier for both automatic and human evaluations.We use Yelp (Shen et al., 2017) and News Titles (Fu et al., 2018) for experiments.Yelp is a collection of restaurant reviews.
We use the pre-processed version used in (Shen et al., 2017), where two polarity sentiment labels are provided.For News Titles, we choose the titles belong to Business, Entertainment and Health categories for our experiments.Both Yelp and News Titles are datasets with relatively short text.We filter out text longer than 15 words, then choose the top 8,900 and 10,000 words as the vocabulary for Yelp and News Titles, respectively.The statistics of the two datasets are listed in Table 1.We discard the labels in the original training and validation splits.We use the original training split as the unlabeled corpus; the validation split to select the best unsupervised models, and the test split as the labeled conditional text.
Based on the Yelp dataset, we define two tasks: (1) Sentiment.This task aims at generating text samples, either positive or negative.The ratio of positive/negative text in Yelp is roughly 0.6 : 0.4.We randomly sample 200 positive and 200 negative text for supervised training.(2) Length.This task aims at generating text samples with a specific length.We define (len ≤ 3) as short text, (len ≥ 12) as long text and (3 < len < 12) as medium text.We respectively sample 200 text for short, medium, and long text for supervised training.
Based on the News Titles dataset, we define the categorical text generation task called Topic.This task aims at generating text samples on a certain topic.The ratio of business/health/entertainment in News Title is 0.38 : 0.15 : 0.47, which is more imbalanced than Yelp.We randomly sample 200 text for each category for supervised learning.

Baselines
We use two semi-supervised methods, S-VAE (Kingma et al., 2014) and CTRL-GEN (Hu et al., 2017) as our baselines.S-VAE incorporates a classifier to provide conditional distribution for unlabeled data.Note that S-VAE is originally proposed for image generation but adapted to text generation as a baseline by Hu et al. (2017).CTRL-GEN further exploits several regularization terms to enhance the disentanglement between the structured code and the unstructured code.For a fair comparison, both the text encoder and decoder of the two baselines are the same as PRETRAINVAE.Furthermore, the baseline methods also exploit the same unlabeled corpus X and labeled corpus Y as described in the original papers.

Models
PPVAE is a model-agnostic approach, which means that both the encoders and encoders of PRE-TRAINVAE and PLUGINVAE can be modified to work under different settings.Here, we describe the model architecture used in our experiments.PRETRAINVAE.For the encoder, we use a onelayer Bidirectional Gated Recurrent Unit (Bi-GRU) with 256 hidden units in each direction as its encoder.Two linear Fully-Connected (FC) layers are used for re-parameteristic trick (Kingma and Welling, 2014).For the decoder, we use a Transformer (Vaswani et al., 2017) (3 layers, 8 heads).Additionally, we add extra positional embedding after each block, and the linearly transformed encoded vector is provided as input for each block (Brock et al., 2019).For a fair comparison, we use the same encoder-decoder architecture for both S-VAE and CTRL-GEN.PLUGINVAE.The encoder is a two-layer FC network of 64/32 hidden units taking input in d g dimensions with an additional linear output layer of d c units.The decoder is a two-layer FC network of 32/64 hidden units taking the latent variable in d c dimensions as input with a linear output layer of d g units.The activation function used in the FC networks is LeakyRelu (Maas et al., 2013).

Hyper-Parameters
PRETRAINVAE.The size of latent space d g is set to 128.The word embedding is in 256 dimensions and randomly initialized.The output softmax matrix is tied with the embedding layer.For the adversarial classifier, we adopt two 128D hidden FC layers with LeakyRelu activation and one 1D output linear layer without bias.The balance coefficient λ is 20 for Yelp and 15 for News Titles.We train the WAE-GAN with Wasserstein Divergence (Wu et al., 2018) 2015) with beta 1 = 0 is used as the optimizer.The learning rate is set to 5 × 10 −4 .PLUGINVAE.We set the size of latent space d c = 20.γ is set to 0.1 for sentiment tasks, 0.05 for categorical tasks, and 3 × 10 −3 for length tasks.The batch size is set to 128.Adam (Kingma and Ba, 2015) with beta 1 = 0.5 is used as the optimizer, learning rate is 3 × 10 −4 for 20K iterations.β linearly increases from 0 to 5 in first 10K iterations.

Evaluation Settings
Metrics.We evaluate the results with two metrics, accuracy and diversity.For accuracy, we train a sentiment classifier and categorical classifier (Kim, 2014), which could achieve accuracy of 90% and 97% on the validation set, respectively.The accuracy of length task can be directly calculated with the word count of generated text.Plus, a model that performs well on only one condition but poorly on others is not practically useful.Thus, to measure the robustness among conditions, we calculate the variance of accuracy under all conditions in a task.
For diversity, we adopt Distinct-1 and Distinct-2 (Li et al., 2016) metrics.Distinct-1/Distinct-2 are the ratios of unique 1-gram/2-gram, respectively.A higher value indicates better diversity.For all tasks and models, we randomly generate 10K text for each condition by greedy decoding and report the averaged results.Single Condition Generation.In a real-world scenario, the full set of conditions is not always avail-able.When provided only a labeled set of target text (i.e., k = 1), it is not possible to learn the joint conditional space for S-VAE and CTRL-GEN any more.However, PPVAE can deal with that by training without negative samples using Equation 3.
6 Experimental Results

Overall Comparisons
Accuracy.The results of conditional text generation are listed in Table 2. On sentiment task, our model outperforms CTRL-GEN and S-VAE by 0.1486 and 0.129, respectively.On length task, the accuracy of our model exceeds 95%, dramatically outperforming S-VAE and CTRL-GEN by 0.1124 and 0.5765 on accuracy.Notably, the performance of CTRL-GEN (0.3957) is extremely low, demonstrating the limitation of its generatordiscriminator (Goodfellow et al., 2014) training process and its token-based discriminator, which is unable to discriminate text with different lengths.
On topic task, our model scores higher on accuracy than S-VAE and CTRL-GEN by 0.1094 and 0.2689, respectively.On all three tasks, PPVAE-single performs slightly poorer than PPVAE with negative samples, verifying the effectiveness of negative sampling.Furthermore, our models achieve the lowest variance on all three tasks, indicating that PPVAE is robust and achieves a good balance among conditions.Diversity.Diversity is a long-lasting issue lying in the field of generative models.Recent works (Wang et al., 2017;Razavi et al., 2019)

Human Evaluation
We conduct human annotations as a complementary evaluation beyond automatic metrics.Specifically, eight individual judges are asked to rate over 200 conditional samples generated from each model and each condition.That is, for each model, a total of 4, 800 text samples are annotated.A judge needs to rate fluency and conditionality in the standard 1 to 5 scale.Fluency measures whether the text samples are natural and fluent as real (i.e., humanwritten) ones.Conditionality indicates whether the generated text adheres to the given condition.Shown in Table 3, PPVAE achieves the best conditionality in both automatic and human evaluations on all three tasks.Meanwhile, PPVAE retains a satisfying fluency on sentiment and length tasks and obtains the best fluency on the topic task.

Training Costs
To measure the efficiency of proposed methods, we report the training time and the number of parameters of S-VAE, CTRL-GEN and PPVAE in Table 4.We train the models on a single Nvidia   GTX 1080 GPU and report the training time until the convergence of each model.PRETRAINVAE has the same size of S-VAE but only needs to be trained once and does not require a full retraining when a new condition added.Also, PLUGINVAE, which learns to transform between the global latent space and the conditional latent space, only has 22K parameters and can be trained within about one minute.

PLUGINVAE vs. Fine-Tuning
As a natural baseline, the conditional generation can also be done by directly fine-tuning PRETRAIN-VAE on each condition.Shown in Table 5, despite the fact that it is not computationally efficient and saving the full weights is undesirable for industrial applications when the model is large (e.g., GPT-2 (Radford et al., 2019)), both PLUGINVAE trained with and without negative samples significantly outperform a directly fine-tuned PRETRAINVAE on accuracy.

Effect of Hyper-parameter β
Since β is an important hyper-parameter for PP-VAE, we test β ∈ {0, 2, 5, 10} on the long text generation task.From the results in Table 6, we find that β controls the balance between diversity and accuracy.Specifically, when β is too large, more diverse samples could be generated, but the accuracy may be sacrificed slightly.On the contrary, when β is too small, the accuracy could climb to a higher value, but meanwhile, the diversity drops drastically.Empirically, we find that β = 5 is an appropriate value for all tasks.

Case Study
We select some generated conditional text of each condition in Table 7.As shown in the table, our proposed PPVAE is capable of generating realistic conditional text.Also, shown in Table 8, on topic task, we randomly select some examples from the output of each model.The output of S-VAE seems to be diverse but is poorly conditioned.CTRL-GEN suffers an obvious diversity issue, which makes it repeatedly output similar text.
For the error analysis, we pick some failed examples of PPVAE in Table 9.We categorize the errors into two main classes.(1) Grammatical.Grammatical problems are common in NLG.As we analyze, this kind of errors can be mitigated with a deeper encoder and decoder with even more unlabeled data for pre-training.(2) Conditional.Conditional errors are of great interest to us since they lie in our focus.We choose three typical errors and list them in Table 9.In the first sentence, "shocked" is a subtle word which may indicate either positive or negative sentiment depending on the context.Thus, with a greedy decoding strategy, it may be incorrectly decoded into the other polarity.We believe this kind of errors could be fixed with more elaborate decoding strategies (e.g., Weighted Decoding (See et al., 2019)).In the second sentence, the length is limited by the nature of an interrogative sentence.As a linguistic fact, an interrogative sentence often has fewer words than a declarative sentence.In the third sentence, we remark an overlapping problem between classes.Some topics (e.g., music album) may appear in both business and entertainment news.In some way, these samples can also be considered as correctly conditioned ones, which highlights the importance of a fine-grained human evaluation on this task.
In this paper, we present a novel PPVAE framework for flexible conditional text generation, which decouples the text generation module from the condition representation module.The extensive experiments demonstrate the superiority of the proposed PPVAE against the existing alternatives on conditionality and diversity while allowing new conditions to be added without a full retraining.
t e x i t s h a 1 _ b a s e 6 4 = " z e

Figure 1 :
Figure 1: The whole workflow of our proposed framework.
an unlabeled corpus X, and conditional text samplesY = Y 1 ∪ Y 2 ∪ ... ∪ Y k where each Y i is a set of text samples that carries the condition c i .The goal of a VAE model is to learn a decoder p θ (ŷ|z, c i ) that takes the latent variable z and the condition c i to calculate the distribution over the text samples Y i .Thus, when the condition c i and a randomly sampled latent variable z ∼ p(z) specified, the model could generate realistic text samples matching the given condition.4Pre-trainand Plug-in Variational Auto-EncoderAs a basis for semi-supervised learning, a large unlabeled corpus should include diverse text which covers a vast spectrum of conditions.Thus, text under each condition forms a conditional latent space, which could be mapped from a larger global latent space.Based on this, we propose a PRETRAIN-VAE and a PLUGINVAE to derive the global and conditional latent space, respectively.

Table 1 :
The statistics of Yelp and News Titles.
to smooth the training process.The coefficient k and power p of Wasserstein Divergence

Table 2 :
The results of conditional text generation tasks.We use boldface and underline to indicate the best and the second-best performance.PPVAE-single indicates PPVAE with a PLUGINVAE trained under the single condition setting, as described in Section 5.5.We show the natural logarithm (ln) of variance, since the original scale is too small for demonstration.are set to 2 and 6, respectively.During pre-training, the batch size is set to 512.Adam (Kingma and Ba, reveal the capability of the diverse content generation with

Table 3 :
Human evaluation results.Note that since the length task is objectively defined, we copy the accuracy results from Table2.

Table 4 :
Average numbers of parameters and time costs for training.

Table 5 :
The comparisons of fine-tuned PRETRAIN-VAE with the full PPVAE on the two tasks of Yelp dataset.

Table 6 :
The impact of different β on long text generation task.

Table 7 :
work with you and this place.Long And this made me feel uncomfortable and the prices aren't right.Virus Spreads in the US Entertainment Surprise Birthday: The Guys of the Cast of Disney Parks Some conditional examples generated by PPVAE for qualitative analysis (cherry-picked).Chinese State Media: 17 Miners Trapped Underground Huge Increases in Obamacare Premiums Are Coming Herbalife Ltd. (HLF) Probe Earns Bill Ackman Back Millions CTRL-GEN Pfizer's Astrazeneca's Astrazeneca Bid for FDA's New Drug to Treat Migraines Pfizer to Acquire Seragon in $42.9B PPVAE Despite Highway Crisis, Many Worries Remain on US Oil Exports Lululemon: Digital Sales Surge in 1Q Net Income, Revenue Crisis of Market: US Stocks Climb; Nike Jumps

Table 8 :
Some generated conditional examples under condition Business (randomly sampled).I was shocked that this is what I needed.(Long) Are you actually drunk outside?(Business) Michael Jackson's New Album 'Xscape'