论文标题
PVO：全景视觉探光仪
PVO: Panoptic Visual Odometry
论文作者
论文摘要
我们提出了PVO，这是一种新型的全景视觉探针框架，以实现场景运动，几何形状和泛型分割信息的更全面的建模。我们的PVO在统一的视图中模型的视觉进程（VO）和视频全景分割（VPS），这使得这两个任务互惠互利。具体来说，我们在图像泛型分段的指导下将泛型更新模块引入了VO模块。这种泛型增强的VO模块可以通过泛型感知的动态掩码来减轻摄像头姿势估计中动态对象的影响。另一方面，使用摄像机姿势，深度和从VO模块获得的几何信息（例如，vo增强VPS模块）可以通过将当前帧的全面分割结果融合到相邻框架上，从而提高了分割精度。这两个模块通过反复的迭代优化互相贡献。广泛的实验表明，PVO在视觉景观和视频泛型分割任务中的最先进方法均优于最先进的方法。
We present PVO, a novel panoptic visual odometry framework to achieve more comprehensive modeling of the scene motion, geometry, and panoptic segmentation information. Our PVO models visual odometry (VO) and video panoptic segmentation (VPS) in a unified view, which makes the two tasks mutually beneficial. Specifically, we introduce a panoptic update module into the VO Module with the guidance of image panoptic segmentation. This Panoptic-Enhanced VO Module can alleviate the impact of dynamic objects in the camera pose estimation with a panoptic-aware dynamic mask. On the other hand, the VO-Enhanced VPS Module also improves the segmentation accuracy by fusing the panoptic segmentation result of the current frame on the fly to the adjacent frames, using geometric information such as camera pose, depth, and optical flow obtained from the VO Module. These two modules contribute to each other through recurrent iterative optimization. Extensive experiments demonstrate that PVO outperforms state-of-the-art methods in both visual odometry and video panoptic segmentation tasks.
