论文标题
随时用蒸馏层的层次神经合奏推断
Anytime Inference with Distilled Hierarchical Neural Ensembles
论文作者
论文摘要
深度神经网络中的推论在计算上可能很昂贵，并且能够随时推断的网络在MSCENARIOS中很重要，在Mscenarios中，输入数据的计算数量或数量随时间而变化。在这样的网络中，推理过程可以中断以更快地提供结果，或者继续获得更准确的结果。我们提出了分层神经合奏（HNE），这是一个新颖的框架，将多个网络的集合嵌入分层树结构中，共享中间层。在HNE中，我们通过评估或多或少的合奏模型来控制推理的复杂性。我们的第二个贡献是一种新型的分层蒸馏方法，可提高小合奏的预测准确性。这种方法利用了我们的合奏的嵌套结构，以最佳分配各个模型的准确性和多样性。我们的实验表明，与以前的任何时间推理模型相比，HNE在CIFAR-10/100和Imagenet数据集上提供了最先进的准确性计算折衷。
Inference in deep neural networks can be computationally expensive, and networks capable of anytime inference are important in mscenarios where the amount of compute or quantity of input data varies over time. In such networks the inference process can interrupted to provide a result faster, or continued to obtain a more accurate result. We propose Hierarchical Neural Ensembles (HNE), a novel framework to embed an ensemble of multiple networks in a hierarchical tree structure, sharing intermediate layers. In HNE we control the complexity of inference on-the-fly by evaluating more or less models in the ensemble. Our second contribution is a novel hierarchical distillation method to boost the prediction accuracy of small ensembles. This approach leverages the nested structure of our ensembles, to optimally allocate accuracy and diversity across the individual models. Our experiments show that, compared to previous anytime inference models, HNE provides state-of-the-art accuracy-computate trade-offs on the CIFAR-10/100 and ImageNet datasets.
