论文标题
自我监督360 $^{\ circ} $房间布局估算
Self-supervised 360$^{\circ}$ Room Layout Estimation
论文作者
论文摘要
我们提出了第一个自我监督的方法，用于训练全景房间布局估计模型，而没有任何标记的数据。与提供丰富的对应关系约束的人均密度深度不同，布局表示稀疏且拓扑，阻碍了图像上自我监督的重新注射一致性的使用。为了解决此问题，我们提出了可区分的布局视图渲染，可以将源图像扭曲到目标摄像头姿势，鉴于来自目标图像的估计布局。由于每个渲染像素相对于估计的布局都是可区分的，因此我们现在可以通过最大程度地减少再投影损失来训练布局估计模型。此外，我们引入正规化损失，以鼓励曼哈顿对齐，天花板对准，周期一致性和布局伸展一致性，从而进一步改善了我们的预测。最后，我们在Zilloindoor和MatterPortlayout数据集上介绍了第一个自我监督的结果。我们的方法还显示了在数据筛选方案和主动学习中有希望的解决方案，这将在房地产虚拟旅游软件中具有直接的价值。代码可在https://github.com/joshua049/stereo-360-layout上找到。
We present the first self-supervised method to train panoramic room layout estimation models without any labeled data. Unlike per-pixel dense depth that provides abundant correspondence constraints, layout representation is sparse and topological, hindering the use of self-supervised reprojection consistency on images. To address this issue, we propose Differentiable Layout View Rendering, which can warp a source image to the target camera pose given the estimated layout from the target image. As each rendered pixel is differentiable with respect to the estimated layout, we can now train the layout estimation model by minimizing reprojection loss. Besides, we introduce regularization losses to encourage Manhattan alignment, ceiling-floor alignment, cycle consistency, and layout stretch consistency, which further improve our predictions. Finally, we present the first self-supervised results on ZilloIndoor and MatterportLayout datasets. Our approach also shows promising solutions in data-scarce scenarios and active learning, which would have an immediate value in the real estate virtual tour software. Code is available at https://github.com/joshua049/Stereo-360-Layout.
