论文标题
通过高斯流程先验的变异编码器模型的多样化文本生成
Diverse Text Generation via Variational Encoder-Decoder Models with Gaussian Process Priors
论文作者
论文摘要
生成具有高度多样性的高质量文本对于许多NLG应用程序很重要，但是当前的方法主要集中于构建确定性模型以生成更高质量的文本，并且没有为促进多样性提供许多选择。在这项工作中，我们提出了一种新型的潜在结构变量模型，以通过丰富编码器模型的上下文表示学习来生成高质量的文本。具体而言，我们引入了一个随机函数，将确定性编码器隐藏状态映射到随机上下文变量中。在（1）提供无限数量的随机上下文变量（多样性促进）和（2）（2）在上下文变量之间明确建模的依赖性（准确编码）之前，从高斯过程中取样了提出的随机函数。为了应对高斯过程的学习挑战，我们提出了一种有效的变分推断方法，以近似随机上下文变量的后验分布。我们在两个典型的文本生成任务中评估我们的方法：释义生成和文本样式转移。基准数据集的实验结果表明，与其他基准相比，我们的方法可以提高发电质量和多样性。
Generating high quality texts with high diversity is important for many NLG applications, but current methods mostly focus on building deterministic models to generate higher quality texts and do not provide many options for promoting diversity. In this work, we present a novel latent structured variable model to generate high quality texts by enriching contextual representation learning of encoder-decoder models. Specifically, we introduce a stochastic function to map deterministic encoder hidden states into random context variables. The proposed stochastic function is sampled from a Gaussian process prior to (1) provide infinite number of joint Gaussian distributions of random context variables (diversity-promoting) and (2) explicitly model dependency between context variables (accurate-encoding). To address the learning challenge of Gaussian processes, we propose an efficient variational inference approach to approximate the posterior distribution of random context variables. We evaluate our method in two typical text generation tasks: paraphrase generation and text style transfer. Experimental results on benchmark datasets demonstrate that our method improves the generation quality and diversity compared with other baselines.
