论文标题
蒙面的贝叶斯神经网络：计算和最佳性
Masked Bayesian Neural Networks : Computation and Optimality
论文作者
论文摘要
随着数据大小和计算能力的增加，深神经网络（DNN）的体系结构变得越来越复杂和庞大，因此，越来越需要简化这种复杂且巨大的DNN。在本文中，我们提出了一种新型的稀疏贝叶斯神经网络（BNN），该网络（BNN）搜索具有适当复杂性的良好DNN。我们在每个节点上采用掩蔽变量，可以根据后分布关闭某些节点以产生略有稀疏的DNN。我们设计了先前的分布，使后验分布具有理论上的最佳性（即最小值的最佳性和适应性），并开发有效的MCMC算法。通过分析几个基准数据集，我们说明了所提出的BNN与其他现有方法相比，其性能很好，即它发现具有相似的预测准确性和与大型DNN相比的预测准确性和不确定性量化的良好凝结的DNN体系结构。
As data size and computing power increase, the architectures of deep neural networks (DNNs) have been getting more complex and huge, and thus there is a growing need to simplify such complex and huge DNNs. In this paper, we propose a novel sparse Bayesian neural network (BNN) which searches a good DNN with an appropriate complexity. We employ the masking variables at each node which can turn off some nodes according to the posterior distribution to yield a nodewise sparse DNN. We devise a prior distribution such that the posterior distribution has theoretical optimalities (i.e. minimax optimality and adaptiveness), and develop an efficient MCMC algorithm. By analyzing several benchmark datasets, we illustrate that the proposed BNN performs well compared to other existing methods in the sense that it discovers well condensed DNN architectures with similar prediction accuracy and uncertainty quantification compared to large DNNs.
