论文标题
层合奏
Layer Ensembles
论文作者
论文摘要
作为贝叶斯神经网络的一种深层合奏，可以通过从每个网络中收集投票并计算这些预测的差异来估计多个神经网络预测的不确定性。在本文中，我们介绍了一种不确定性估计的方法，该方法考虑了网络的每个层的一组独立的分类分布，提供了与常规深层合奏相比，具有多层层的许多可能的样本。我们进一步介绍了一个优化的推理过程，该过程重用公共层输出，最多达到19倍的速度并四处降低内存使用情况。我们还表明，可以通过对样本进行排名，从而进一步改进该方法，从而导致模型需要更少的内存和时间来运行，同时比深层集合更高的不确定性质量。
Deep Ensembles, as a type of Bayesian Neural Networks, can be used to estimate uncertainty on the prediction of multiple neural networks by collecting votes from each network and computing the difference in those predictions. In this paper, we introduce a method for uncertainty estimation that considers a set of independent categorical distributions for each layer of the network, giving many more possible samples with overlapped layers than in the regular Deep Ensembles. We further introduce an optimized inference procedure that reuses common layer outputs, achieving up to 19x speed up and reducing memory usage quadratically. We also show that the method can be further improved by ranking samples, resulting in models that require less memory and time to run while achieving higher uncertainty quality than Deep Ensembles.
