论文标题
基于神经压缩的功能学习视频修复
Neural Compression-Based Feature Learning for Video Restoration
论文作者
论文摘要
如何有效利用时间特征对于视频修复至关重要，但具有挑战性。时间特征通常包含各种嘈杂和不相关的信息，它们可能会干扰当前框架的恢复。本文提出了学习噪音般的功能表示形式，以帮助视频恢复。我们的灵感来自神经编解码器是天然的Deoiser。在神经编解码器中，很难预测的嘈杂和不相关的内容，但成本很多更倾向于丢弃以节省比特率。因此，我们设计了一个神经压缩模块，以过滤噪声并将最有用的信息保留在视频修复功能中。为了实现噪声的鲁棒性，我们的压缩模块采用了空间通道量化机制，以适应潜在位置的每个位置的量化步长。实验表明，我们的方法可以显着提高视频deNoising的性能，在此，我们仅使用0.23倍的拖鞋获得了0.13 dB的改进。同时，我们的方法还获得了视频降低和飞行的SOTA结果。
How to efficiently utilize the temporal features is crucial, yet challenging, for video restoration. The temporal features usually contain various noisy and uncorrelated information, and they may interfere with the restoration of the current frame. This paper proposes learning noise-robust feature representations to help video restoration. We are inspired by that the neural codec is a natural denoiser. In neural codec, the noisy and uncorrelated contents which are hard to predict but cost lots of bits are more inclined to be discarded for bitrate saving. Therefore, we design a neural compression module to filter the noise and keep the most useful information in features for video restoration. To achieve robustness to noise, our compression module adopts a spatial channel-wise quantization mechanism to adaptively determine the quantization step size for each position in the latent. Experiments show that our method can significantly boost the performance on video denoising, where we obtain 0.13 dB improvement over BasicVSR++ with only 0.23x FLOPs. Meanwhile, our method also obtains SOTA results on video deraining and dehazing.
