论文标题
部分可观测时空混沌系统的无模型预测
Pitfalls of Gaussians as a noise distribution in NCE
论文作者
论文摘要
噪声对比估计（NCE）是一种流行的学习概率密度函数的流行方法，直至相称的常数。主要思想是设计一个分类问题，以将训练数据与易于样本噪声分布$ Q $区分开来，以避免计算分区功能的方式。众所周知，$ Q $的选择会严重影响NCE的计算和统计效率。实际上，$ Q $的共同选择是与数据的平均值和协方差相匹配的高斯。 在本文中，我们表明，即使对于非常简单的数据分布，这种选择也可能导致损失的黑森的指数差（在环境维度）条件。结果，对于$ Q $选择，统计和算法复杂性在实践中都是有问题的，这表明更复杂的噪声分布对于NCE的成功至关重要。
Noise Contrastive Estimation (NCE) is a popular approach for learning probability density functions parameterized up to a constant of proportionality. The main idea is to design a classification problem for distinguishing training data from samples from an easy-to-sample noise distribution $q$, in a manner that avoids having to calculate a partition function. It is well-known that the choice of $q$ can severely impact the computational and statistical efficiency of NCE. In practice, a common choice for $q$ is a Gaussian which matches the mean and covariance of the data. In this paper, we show that such a choice can result in an exponentially bad (in the ambient dimension) conditioning of the Hessian of the loss, even for very simple data distributions. As a consequence, both the statistical and algorithmic complexity for such a choice of $q$ will be problematic in practice, suggesting that more complex noise distributions are essential to the success of NCE.
