论文标题
训练前的单枪结构修剪
Single Shot Structured Pruning Before Training
论文作者
论文摘要
我们介绍了一种方法，以使用在训练前应用的结构化修剪来加快2倍的训练，并在深度神经网络中通过3倍推理。与以前关于修剪训练的训练的工作不同，我们的工作开发了一种方法，以删除整个渠道和隐藏单元的方法，以加快培训和推理的明确目的。我们引入了一种计算意识的评分机制，该机制可以以每次翻牌的灵敏度进行修剪，从而使速度更高。我们的方法快速，易于实现，并且在训练开始之前只需要一批数据上的一个前向/向后传递即可完成修剪。
We introduce a method to speed up training by 2x and inference by 3x in deep neural networks using structured pruning applied before training. Unlike previous works on pruning before training which prune individual weights, our work develops a methodology to remove entire channels and hidden units with the explicit aim of speeding up training and inference. We introduce a compute-aware scoring mechanism which enables pruning in units of sensitivity per FLOP removed, allowing even greater speed ups. Our method is fast, easy to implement, and needs just one forward/backward pass on a single batch of data to complete pruning before training begins.
