论文标题
培训具有实际二元卷积的二进制神经网络
Training Binary Neural Networks with Real-to-Binary Convolutions
论文作者
论文摘要
本文展示了如何将二进制网络训练到完整的Precision对应物的几个百分点（$ \ sim 3-5 \％$）之内。我们首先展示了如何通过结合最近提出的进步并仔细调整优化程序来构建强大的基线，该基线已经实现了最先进的准确性。其次，我们表明，通过尝试最大程度地减少二进制输出与相应的实价卷积之间的差异，可以获得更多的显着准确性提高。我们以两种互补的方式实现了这一想法：（1）通过损失功能，在训练期间，通过使用二进制和实地值的卷积的输出计算的空间注意力映射，以及（2）通过使用真实价值的激活来进行重新估算的重新计算，以数据驱动的方式使用真实价值的激活，可用于重新估算的重新估算。最后，我们表明，当我们将所有改进放在一起时，提议的模型在Imagenet上以超过5％的top-1精度击败了目前的艺术状态，并将其差距降低到其实用值的差距为CIFAR-100和IMAGENET上分别使用Resnet-18架构时，分别为3％和5％的TOP-1准确性。代码可在https://github.com/brais-martinez/real2binary中找到。
This paper shows how to train binary networks to within a few percent points ($\sim 3-5 \%$) of the full precision counterpart. We first show how to build a strong baseline, which already achieves state-of-the-art accuracy, by combining recently proposed advances and carefully adjusting the optimization procedure. Secondly, we show that by attempting to minimize the discrepancy between the output of the binary and the corresponding real-valued convolution, additional significant accuracy gains can be obtained. We materialize this idea in two complementary ways: (1) with a loss function, during training, by matching the spatial attention maps computed at the output of the binary and real-valued convolutions, and (2) in a data-driven manner, by using the real-valued activations, available during inference prior to the binarization process, for re-scaling the activations right after the binary convolution. Finally, we show that, when putting all of our improvements together, the proposed model beats the current state of the art by more than 5% top-1 accuracy on ImageNet and reduces the gap to its real-valued counterpart to less than 3% and 5% top-1 accuracy on CIFAR-100 and ImageNet respectively when using a ResNet-18 architecture. Code available at https://github.com/brais-martinez/real2binary.
