论文标题
贝叶斯神经网络的数据子采样
Data Subsampling for Bayesian Neural Networks
论文作者
论文摘要
Markov Chain Monte Carlo（MCMC）算法对于大型数据集的扩展不是很好，这会导致神经网络后采样困难。在本文中，我们提出了惩罚贝叶斯神经网络-PBNNS，作为一种新算法，可以在贝叶斯推理上下文中使用亚采样批处理数据（小批量）评估可能性，以解决可伸缩性。 PBNN避免了其他幼稚子采样技术固有的偏见，通过将罚款纳入大都会黑斯廷斯算法的概括的一部分。我们表明，将PBNN与现有的MCMC框架集成很简单，因为损失函数的差异仅降低了接受概率。通过与合成数据和MNIST数据集的替代抽样策略进行比较，我们证明了PBNN即使对于小型小批量数据，PBNN也可以达到良好的预测性能。我们表明，PBNN提供了一种新颖的方法，可以通过改变迷你批量大小来校准预测分布，从而显着降低预测性过高的自信。
Markov Chain Monte Carlo (MCMC) algorithms do not scale well for large datasets leading to difficulties in Neural Network posterior sampling. In this paper, we propose Penalty Bayesian Neural Networks - PBNNs, as a new algorithm that allows the evaluation of the likelihood using subsampled batch data (mini-batches) in a Bayesian inference context towards addressing scalability. PBNN avoids the biases inherent in other naive subsampling techniques by incorporating a penalty term as part of a generalization of the Metropolis Hastings algorithm. We show that it is straightforward to integrate PBNN with existing MCMC frameworks, as the variance of the loss function merely reduces the acceptance probability. By comparing with alternative sampling strategies on both synthetic data and the MNIST dataset, we demonstrate that PBNN achieves good predictive performance even for small mini-batch sizes of data. We show that PBNN provides a novel approach for calibrating the predictive distribution by varying the mini-batch size, significantly reducing predictive overconfidence.
