论文标题
Dirichlet修剪神经网络压缩
Dirichlet Pruning for Neural Network Compression
论文作者
论文摘要
我们介绍了Dirichlet Pruning，这是一种新颖的后处理技术，可将大型神经网络模型转变为一种压缩的技术。 Dirichlet修剪是一种结构化修剪的一种形式，它在卷积层（或完全连接的层中的神经元）中分配了dirichlet分布，并使用各种推断估算了这些单元上分布的参数。学习的分布使我们能够删除不重要的单元，从而导致紧凑的体系结构仅包含手头任务的关键功能。新引入的dirichlet参数的数量仅在通道数中线性，这允许快速训练，只需要一个时期即可收敛。我们进行了广泛的实验，尤其是在较大的架构上，例如VGG和Resnet（分别为45％和58％的压缩率），其中我们的方法实现了最新的压缩性能，并提供了可解释的特征作为副产品。
We introduce Dirichlet pruning, a novel post-processing technique to transform a large neural network model into a compressed one. Dirichlet pruning is a form of structured pruning that assigns the Dirichlet distribution over each layer's channels in convolutional layers (or neurons in fully-connected layers) and estimates the parameters of the distribution over these units using variational inference. The learned distribution allows us to remove unimportant units, resulting in a compact architecture containing only crucial features for a task at hand. The number of newly introduced Dirichlet parameters is only linear in the number of channels, which allows for rapid training, requiring as little as one epoch to converge. We perform extensive experiments, in particular on larger architectures such as VGG and ResNet (45% and 58% compression rate, respectively) where our method achieves the state-of-the-art compression performance and provides interpretable features as a by-product.
