论文标题
整体过滤器修剪，以进行有效的深神经网络
Holistic Filter Pruning for Efficient Deep Neural Networks
论文作者
论文摘要
深度神经网络（DNN）通常被过度参数化，以增加通过随机初始化获得足够初始权重的可能性。因此，训练有素的DNN具有许多冗余，可以从模型中修剪以降低复杂性并提高概括能力。通过过滤器修剪实现的结构稀疏性直接降低了权重和激活的张量，因此对于降低复杂性特别有效。我们提出了“整体过滤器修剪”（HFP），这是一种通用DNN训练的新方法，易于实现，并可以为参数和乘法的数量指定准确的修剪率。每次向前通过后，计算当前模型的复杂性并将其与所需的目标尺寸进行比较。通过梯度下降，可以找到一种全球解决方案，该解决方案将修剪预算分配到各个层，以便满足所需的目标大小。在各种实验中，我们对CIFAR-10和IMAGENET的最先进性能进行洞察力（HFP修剪Imainet-50乘法的60％的ImaTEnet上的乘法占60％，而精度没有明显的损失）。我们认为，我们简单而强大的修剪方法可以为低成本应用中的DNN用户做出宝贵的贡献。
Deep neural networks (DNNs) are usually over-parameterized to increase the likelihood of getting adequate initial weights by random initialization. Consequently, trained DNNs have many redundancies which can be pruned from the model to reduce complexity and improve the ability to generalize. Structural sparsity, as achieved by filter pruning, directly reduces the tensor sizes of weights and activations and is thus particularly effective for reducing complexity. We propose "Holistic Filter Pruning" (HFP), a novel approach for common DNN training that is easy to implement and enables to specify accurate pruning rates for the number of both parameters and multiplications. After each forward pass, the current model complexity is calculated and compared to the desired target size. By gradient descent, a global solution can be found that allocates the pruning budget over the individual layers such that the desired target size is fulfilled. In various experiments, we give insights into the training and achieve state-of-the-art performance on CIFAR-10 and ImageNet (HFP prunes 60% of the multiplications of ResNet-50 on ImageNet with no significant loss in the accuracy). We believe our simple and powerful pruning approach to constitute a valuable contribution for users of DNNs in low-cost applications.
