论文标题
通过自我监督的深度估计来改善语义细分的三种方法
Three Ways to Improve Semantic Segmentation with Self-Supervised Depth Estimation
论文作者
论文摘要
训练深层网络进行语义细分需要大量标记的培训数据，这在实践中提出了一个主要的挑战，因为标记分段口罩是一个高度劳动力密集的过程。为了解决这个问题，我们提出了半监督语义分割的框架，该框架通过未标记的图像序列的自我监督的单眼深度估计来增强。特别是，我们提出了三个关键贡献：（1）我们从在自我监督的深度估算期间学到的特征转移知识到语义细分，（2）我们通过使用场景的几何形状混合图像和标签来实现强大的数据增强，并且（3）我们利用深度多样性的范围以及在学习范围的范围内，以启动范围的范围启动范围，以选择最有用的学生最有效的范围。我们验证了CityScapes数据集上的拟议模型，在该数据集中，这三个模块均显示出显着的性能增长，并且我们获得了半监督语义细分的最新结果。该实现可在https://github.com/lhoyer/improving_segmentation_with_with_selfsupervise_depth上获得。
Training deep networks for semantic segmentation requires large amounts of labeled training data, which presents a major challenge in practice, as labeling segmentation masks is a highly labor-intensive process. To address this issue, we present a framework for semi-supervised semantic segmentation, which is enhanced by self-supervised monocular depth estimation from unlabeled image sequences. In particular, we propose three key contributions: (1) We transfer knowledge from features learned during self-supervised depth estimation to semantic segmentation, (2) we implement a strong data augmentation by blending images and labels using the geometry of the scene, and (3) we utilize the depth feature diversity as well as the level of difficulty of learning depth in a student-teacher framework to select the most useful samples to be annotated for semantic segmentation. We validate the proposed model on the Cityscapes dataset, where all three modules demonstrate significant performance gains, and we achieve state-of-the-art results for semi-supervised semantic segmentation. The implementation is available at https://github.com/lhoyer/improving_segmentation_with_selfsupervised_depth.
