论文标题
贝叶斯神经网络的差异拉普拉斯
Variational Laplace for Bayesian neural networks
论文作者
论文摘要
我们开发了贝叶斯神经网络（BNN）的变异拉普拉斯，该拉普拉斯利用了可能性估算ELBO的局部近似值，而无需对神经网络重量的随机采样。变性拉普拉斯物镜很容易评估，因为它（本质上）是对数可能性的，再加上重量赛，再加上平方级的正规剂。尽管使用了相同的变异后近似值，但差异拉普拉斯与最大A-posteriori推断和基于标准采样的变异推断相比，具有更好的测试性能和预期校准误差。最后，我们强调基准标准VI所需的护理，因为在差异参数融合之前存在停止的风险。我们表明，可以通过提高方差参数的学习率来避免早期停滞。
We develop variational Laplace for Bayesian neural networks (BNNs) which exploits a local approximation of the curvature of the likelihood to estimate the ELBO without the need for stochastic sampling of the neural-network weights. The Variational Laplace objective is simple to evaluate, as it is (in essence) the log-likelihood, plus weight-decay, plus a squared-gradient regularizer. Variational Laplace gave better test performance and expected calibration errors than maximum a-posteriori inference and standard sampling-based variational inference, despite using the same variational approximate posterior. Finally, we emphasise care needed in benchmarking standard VI as there is a risk of stopping before the variance parameters have converged. We show that early-stopping can be avoided by increasing the learning rate for the variance parameters.
