论文标题
软遮罩用于修剪成本约束的频道
Soft Masking for Cost-Constrained Channel Pruning
论文作者
论文摘要
已显示结构化的通道修剪可以显着加速现代硬件上的卷积神经网络（CNN）的推理时间，并且网络准确性的损失相对较小。最近的工作在训练过程中永久归零这些渠道，我们观察到这些渠道会显着妨碍最终准确性，尤其是随着网络的占用比例增加。我们建议对成本约束的通道修剪（SMCP）提出软遮罩，以使修剪的通道可以自适应地返回网络，同时朝目标成本限制进行修剪。通过从删除输入通道的角度添加权重和通道修剪的软蒙版重新参数化，我们允许对先前修剪的通道进行渐变更新，并有机会让渠道以后返回网络。然后，我们将输入通道修剪作为全球资源分配问题。我们的方法在ImageNet分类和Pascal VOC检测数据集上都优于先前工作。
Structured channel pruning has been shown to significantly accelerate inference time for convolution neural networks (CNNs) on modern hardware, with a relatively minor loss of network accuracy. Recent works permanently zero these channels during training, which we observe to significantly hamper final accuracy, particularly as the fraction of the network being pruned increases. We propose Soft Masking for cost-constrained Channel Pruning (SMCP) to allow pruned channels to adaptively return to the network while simultaneously pruning towards a target cost constraint. By adding a soft mask re-parameterization of the weights and channel pruning from the perspective of removing input channels, we allow gradient updates to previously pruned channels and the opportunity for the channels to later return to the network. We then formulate input channel pruning as a global resource allocation problem. Our method outperforms prior works on both the ImageNet classification and PASCAL VOC detection datasets.
