论文标题
统一运动与事件的插值和框架插值
Unifying Motion Deblurring and Frame Interpolation with Events
论文作者
论文摘要
慢速快门速度和基于框架的相机的长时间曝光时间通常会导致视觉模糊和丢失框架间信息，从而脱离了捕获视频的整体质量。为此，我们为模糊视频增强的基于事件的运动去插和框架插值提供了一个统一的框架，其中利用了极低的事件延迟，以减轻运动模糊并促进中间框架预测。具体而言，模糊框架和尖锐的潜在图像之间的映射关系首先是由可学习的双积分网络预测的，然后提出了融合网络，以通过使用连续模糊输入和并发事件的信息来完善粗糙的结果。通过探索模糊框架，潜在图像和事件流之间的相互限制，我们进一步提出了一个自我监管的学习框架，以实现使用现实世界中模糊的视频和事件的网络培训。广泛的实验表明，我们的方法与最先进的方法相比，并在合成和现实世界数据集上取得了显着的性能。
Slow shutter speed and long exposure time of frame-based cameras often cause visual blur and loss of inter-frame information, degenerating the overall quality of captured videos. To this end, we present a unified framework of event-based motion deblurring and frame interpolation for blurry video enhancement, where the extremely low latency of events is leveraged to alleviate motion blur and facilitate intermediate frame prediction. Specifically, the mapping relation between blurry frames and sharp latent images is first predicted by a learnable double integral network, and a fusion network is then proposed to refine the coarse results via utilizing the information from consecutive blurry inputs and the concurrent events. By exploring the mutual constraints among blurry frames, latent images, and event streams, we further propose a self-supervised learning framework to enable network training with real-world blurry videos and events. Extensive experiments demonstrate that our method compares favorably against the state-of-the-art approaches and achieves remarkable performance on both synthetic and real-world datasets.
