论文标题
改进基于能量模型的对比度差异训练
Improved Contrastive Divergence Training of Energy Based Models
论文作者
论文摘要
对比性差异是一种训练基于能量的模型的流行方法，但众所周知，训练稳定性困难。我们提出了一种改编，以审查难以计算的梯度术语来改善对比度差异训练，并且通常是为了方便而被排除在外。我们表明，这个梯度术语具有数值意义，实际上对于避免训练不稳定性很重要，同时可以进行估计。我们进一步强调了如何使用数据增强和多尺度处理来提高模型鲁棒性和发电质量。最后，我们通过经验评估模型架构的稳定性，并在许多基准和用例中显示出改善的性能，例如图像产生，OOD检测和组成产生。
Contrastive divergence is a popular method of training energy-based models, but is known to have difficulties with training stability. We propose an adaptation to improve contrastive divergence training by scrutinizing a gradient term that is difficult to calculate and is often left out for convenience. We show that this gradient term is numerically significant and in practice is important to avoid training instabilities, while being tractable to estimate. We further highlight how data augmentation and multi-scale processing can be used to improve model robustness and generation quality. Finally, we empirically evaluate stability of model architectures and show improved performance on a host of benchmarks and use cases,such as image generation, OOD detection, and compositional generation.
