论文标题
DMCP：神经网络的马尔可夫通道修剪
DMCP: Differentiable Markov Channel Pruning for Neural Networks
论文作者
论文摘要
最近的作品暗示，可以将频道修剪视为未经修复网络的最佳子结构的搜索。但是，基于此观察结果的现有作品需要培训并评估大量结构，这限制了其应用。在本文中，我们提出了一种新颖的可区分方法，用于通道修剪，命名为可区分的马尔可夫通道修剪（DMCP），以有效地搜索最佳子结构。我们的方法是可区分的，可以通过梯度下降直接优化标准任务损失和预算正规化（例如，FLOPS约束）。在DMCP中，我们将通道修剪建模为马尔可夫过程，其中每个状态代表在修剪过程中保留相应的通道，并且状态之间的过渡表示修剪过程。最后，我们的方法能够通过Markov进程通过优化的过渡来隐式地选择每一层中的适当数量的通道数。为了验证我们的方法的有效性，我们对Resnet和MobilenetV2进行了广泛的实验。结果表明，我们的方法可以在各种失败设置中的最先进的修剪方法获得一致的改进。该代码可从https://github.com/zx55/dmcp获得
Recent works imply that the channel pruning can be regarded as searching optimal sub-structure from unpruned networks. However, existing works based on this observation require training and evaluating a large number of structures, which limits their application. In this paper, we propose a novel differentiable method for channel pruning, named Differentiable Markov Channel Pruning (DMCP), to efficiently search the optimal sub-structure. Our method is differentiable and can be directly optimized by gradient descent with respect to standard task loss and budget regularization (e.g. FLOPs constraint). In DMCP, we model the channel pruning as a Markov process, in which each state represents for retaining the corresponding channel during pruning, and transitions between states denote the pruning process. In the end, our method is able to implicitly select the proper number of channels in each layer by the Markov process with optimized transitions. To validate the effectiveness of our method, we perform extensive experiments on Imagenet with ResNet and MobilenetV2. Results show our method can achieve consistent improvement than state-of-the-art pruning methods in various FLOPs settings. The code is available at https://github.com/zx55/dmcp
