论文标题
关于无偏α分歧最小化的难度
On the Difficulty of Unbiased Alpha Divergence Minimization
论文作者
论文摘要
已经提出了几种近似推理算法，以最大程度地减少近似分布和目标分布之间的α差异。这些算法中的许多引入了偏见，其大小在高维度中变得有问题。其他算法是公正的。这些似乎经常遭受较高的差异，但几乎没有广为人知。在这项工作中，我们研究了通过梯度估计量的信噪比（SNR）来最大程度地化α-差异的方法。我们研究了几种代表性的方案，其中可能有强大的分析结果，例如完全成分或高斯分布。我们发现，当alpha不是零时，SNR在问题的维度上成倍恶化。这对这些方法的实用性产生了怀疑。我们从经验上证实了这些理论结果。
Several approximate inference algorithms have been proposed to minimize an alpha-divergence between an approximating distribution and a target distribution. Many of these algorithms introduce bias, the magnitude of which becomes problematic in high dimensions. Other algorithms are unbiased. These often seem to suffer from high variance, but little is rigorously known. In this work we study unbiased methods for alpha-divergence minimization through the Signal-to-Noise Ratio (SNR) of the gradient estimator. We study several representative scenarios where strong analytical results are possible, such as fully-factorized or Gaussian distributions. We find that when alpha is not zero, the SNR worsens exponentially in the dimensionality of the problem. This casts doubt on the practicality of these methods. We empirically confirm these theoretical results.
