论文标题
有效视频表示的神经残留流场
Neural Residual Flow Fields for Efficient Video Representations
论文作者
论文摘要
神经场已成为代表各种信号（包括视频）的强大范式。但是，关于提高神经场参数效率的研究仍处于早期阶段。即使可以使用映射到颜色的神经字段来编码视频信号，但该方案并不能利用视频信号的空间和时间冗余。受标准视频压缩算法的启发，我们提出了一种神经场架构，用于表示和压缩视频，该视频故意通过在视频帧中使用运动信息来故意消除数据冗余。维护运动信息通常更顺畅，比颜色信号更复杂，需要少量的参数。此外，通过运动信息重用颜色值进一步提高了网络参数效率。此外，我们建议对视频框架重建和单独的网络使用多个参考框架，一个用于光流，另一个用于残差。实验结果表明，所提出的方法的表现优于基线方法的明显边缘。该代码可在https://github.com/daniel03c1/eff_video_representation中找到
Neural fields have emerged as a powerful paradigm for representing various signals, including videos. However, research on improving the parameter efficiency of neural fields is still in its early stages. Even though neural fields that map coordinates to colors can be used to encode video signals, this scheme does not exploit the spatial and temporal redundancy of video signals. Inspired by standard video compression algorithms, we propose a neural field architecture for representing and compressing videos that deliberately removes data redundancy through the use of motion information across video frames. Maintaining motion information, which is typically smoother and less complex than color signals, requires a far fewer number of parameters. Furthermore, reusing color values through motion information further improves the network parameter efficiency. In addition, we suggest using more than one reference frame for video frame reconstruction and separate networks, one for optical flows and the other for residuals. Experimental results have shown that the proposed method outperforms the baseline methods by a significant margin. The code is available in https://github.com/daniel03c1/eff_video_representation
