论文标题
WeightMom：使用基于迭代动量的修剪学习稀疏网络
WeightMom: Learning Sparse Networks using Iterative Momentum-based pruning
论文作者
论文摘要
深层神经网络已用于多种成功的应用中。但是，由于包含数百万个参数，它们的高度复杂性质导致在延迟需求低的管道中部署期间有问题。结果，更希望获得在推理期间具有相同性能的轻型神经网络。在这项工作中，我们提出了一种基于体重的修剪方法，其中根据以前的迭代势头逐渐修剪权重。神经网络的每个层都基于其相对稀疏性分配了重要性值，然后在先前的迭代中分配了重量的大小。我们在诸如Alexnet，vgg16和resnet50等网络上评估了我们的方法，其中包括图像分类数据集，例如CIFAR-10和CIFAR-100。我们发现，在准确性和压缩比方面，结果优于先前的方法。我们的方法能够在两个数据集上的同一降解中获得15％的压缩。
Deep Neural Networks have been used in a wide variety of applications with significant success. However, their highly complex nature owing to comprising millions of parameters has lead to problems during deployment in pipelines with low latency requirements. As a result, it is more desirable to obtain lightweight neural networks which have the same performance during inference time. In this work, we propose a weight based pruning approach in which the weights are pruned gradually based on their momentum of the previous iterations. Each layer of the neural network is assigned an importance value based on their relative sparsity, followed by the magnitude of the weight in the previous iterations. We evaluate our approach on networks such as AlexNet, VGG16 and ResNet50 with image classification datasets such as CIFAR-10 and CIFAR-100. We found that the results outperformed the previous approaches with respect to accuracy and compression ratio. Our method is able to obtain a compression of 15% for the same degradation in accuracy on both the datasets.
