论文标题
用于框架插值的稀疏引导网络设计
Sparsity-guided Network Design for Frame Interpolation
论文作者
论文摘要
基于DNN的框架插值从两个连续的帧中生成中间帧，通常取决于具有大量功能的模型体系结构，从而阻止其在具有有限资源的系统（例如移动设备）上部署。我们提出了一种用于框架插值的压缩驱动的网络设计，该设计通过稀疏性诱导优化来利用模型，以大大降低模型大小，同时达到更高的性能。具体而言，我们首先压缩了最近提出的ADACOF模型，并证明了10次压缩ADACOF的性能类似于其原始对应物，在各种超参数设置下，对使用layerwise稀疏信息作为指导的不同策略进行了全面研究。然后，我们通过引入多分辨率翘曲模块来增强这种压缩模型，从而通过多层次的细节提高了视觉一致性。结果，我们在原始AdaCof的四分之一的范围内获得了可观的性能增长。此外，我们的模型在各种数据集上的其他最先进方法都表现出色。我们注意到，建议的压缩驱动的框架是通用的，并且可以轻松地转移到其他基于DNN的框架插值算法中。源代码可在https://github.com/tding1/cdfi上找到。
DNN-based frame interpolation, which generates intermediate frames from two consecutive frames, is often dependent on model architectures with a large number of features, preventing their deployment on systems with limited resources, such as mobile devices. We present a compression-driven network design for frame interpolation that leverages model pruning through sparsity-inducing optimization to greatly reduce the model size while attaining higher performance. Concretely, we begin by compressing the recently proposed AdaCoF model and demonstrating that a 10 times compressed AdaCoF performs similarly to its original counterpart, where different strategies for using layerwise sparsity information as a guide are comprehensively investigated under a variety of hyperparameter settings. We then enhance this compressed model by introducing a multi-resolution warping module, which improves visual consistency with multi-level details. As a result, we achieve a considerable performance gain with a quarter of the size of the original AdaCoF. In addition, our model performs favorably against other state-of-the-art approaches on a wide variety of datasets. We note that the suggested compression-driven framework is generic and can be easily transferred to other DNN-based frame interpolation algorithms. The source code is available at https://github.com/tding1/CDFI.
