论文标题
奇异模型中的证据界限：概率和变异观点
Evidence bounds in singular models: probabilistic and variational perspectives
论文作者
论文摘要
贝叶斯统计数据中的边际可能性或证据包含对较大模型大小的内在惩罚，并且是贝叶斯模型比较的基本数量。在过去的二十年中，在奇异统计模型中，在苏马省渡边的开拓性工作的基础上，人们一直在稳步增加活动，以了解这种惩罚的性质。与常规模型不同的是，贝叶斯信息标准（BIC）封装了边缘可能性的对数的一阶扩展，参数计数在单数模型中变得更棘手，其中称为真实日志规范阈值（RLCT）的数量总结了有效模型尺寸。在本文中，我们提供了一种概率处理，以恢复已建立证据范围的非反应版本，并证明基于吉布斯变化不平等的新结果。特别是，我们表明，平均场变异推理可以正确恢复其规范或正常形式的任何单数模型的RLCT。另外，我们通过分析了一种普遍用于变异推断的通用坐标上升算法（CAVI）的动力学来表现出界限的清晰度。
The marginal likelihood or evidence in Bayesian statistics contains an intrinsic penalty for larger model sizes and is a fundamental quantity in Bayesian model comparison. Over the past two decades, there has been steadily increasing activity to understand the nature of this penalty in singular statistical models, building on pioneering work by Sumio Watanabe. Unlike regular models where the Bayesian information criterion (BIC) encapsulates a first-order expansion of the logarithm of the marginal likelihood, parameter counting gets trickier in singular models where a quantity called the real log canonical threshold (RLCT) summarizes the effective model dimensionality. In this article, we offer a probabilistic treatment to recover non-asymptotic versions of established evidence bounds as well as prove a new result based on the Gibbs variational inequality. In particular, we show that mean-field variational inference correctly recovers the RLCT for any singular model in its canonical or normal form. We additionally exhibit sharpness of our bound by analyzing the dynamics of a general purpose coordinate ascent algorithm (CAVI) popularly employed in variational inference.
