论文标题
Lilnetx：具有极端模型压缩和结构性稀疏的轻量级网络
LilNetX: Lightweight Networks with EXtreme Model Compression and Structured Sparsification
论文作者
论文摘要
我们介绍了Lilnetx，这是一种针对神经网络的端到端训练技术，可以使学习模型具有指定的准确性评估折衷权衡。先前的工作一次解决这些问题，并且通常需要后处理或多阶段培训，这些培训变得不太实际，对于大型数据集或架构而言并不能很好地扩展。我们的方法构建了一个联合培训目标，该目标会在重新聚集的潜在空间中惩罚网络参数的自我信息，以鼓励小型模型大小，同时还引入先验以增加参数空间中的结构性稀疏性以减少计算。与现有的现有目前的最新模型压缩方法相比，我们在Resnet-20上达到了高达50％的型号和98％的模型稀疏性，同时保留了CIFAR-10数据集的相同精度，以及35％较小的型号尺寸和42％的在Imagenet训练的Resnet-50的结构性稀疏性。代码可从https://github.com/sharath-girish/lilnetx获得。
We introduce LilNetX, an end-to-end trainable technique for neural networks that enables learning models with specified accuracy-rate-computation trade-off. Prior works approach these problems one at a time and often require post-processing or multistage training which become less practical and do not scale very well for large datasets or architectures. Our method constructs a joint training objective that penalizes the self-information of network parameters in a reparameterized latent space to encourage small model size while also introducing priors to increase structured sparsity in the parameter space to reduce computation. We achieve up to 50% smaller model size and 98% model sparsity on ResNet-20 while retaining the same accuracy on the CIFAR-10 dataset as well as 35% smaller model size and 42% structured sparsity on ResNet-50 trained on ImageNet, when compared to existing state-of-the-art model compression methods. Code is available at https://github.com/Sharath-girish/LilNetX.
