论文标题
螺母：通过插补和动态编程进行序列建模
Imputer: Sequence Modelling via Imputation and Dynamic Programming
论文作者
论文摘要
本文介绍了螺母，这是一种神经序列模型，该模型通过归纳来迭代产生输出序列。螺旋桨是一种迭代生成模型，仅需要一个恒定数量的生成步骤，而不是输入或输出令牌的数量。可以训练螺旋桨在输入和输出序列以及所有可能的生成顺序之间的所有可能的比对上近似边缘化。我们提出了一种可拖动的动态编程训练算法，该算法在对数边际可能性上产生下限。当应用于端到端的语音识别时，螺旋桨的表现优于先前的非入学模型，并取得了竞争成果，从而获得自回归模型。在librispeech的其他方法上，螺母可以达到11.1，在13.0 wer和seq2seq的表现优于12.5 wer。
This paper presents the Imputer, a neural sequence model that generates output sequences iteratively via imputations. The Imputer is an iterative generative model, requiring only a constant number of generation steps independent of the number of input or output tokens. The Imputer can be trained to approximately marginalize over all possible alignments between the input and output sequences, and all possible generation orders. We present a tractable dynamic programming training algorithm, which yields a lower bound on the log marginal likelihood. When applied to end-to-end speech recognition, the Imputer outperforms prior non-autoregressive models and achieves competitive results to autoregressive models. On LibriSpeech test-other, the Imputer achieves 11.1 WER, outperforming CTC at 13.0 WER and seq2seq at 12.5 WER.
