论文标题
功能中的烘烤：通过渲染特征图加速体积分割
Baking in the Feature: Accelerating Volumetric Segmentation by Rendering Feature Maps
论文作者
论文摘要
最近，已经提出了方法，仅使用稀疏的语义注释像素的形式使用颜色图像和专家监督，将密集的段3D卷成类。尽管令人印象深刻，但这些方法仍然需要相对较大的监督和对象进行分割可能需要几分钟的实践。这样的系统通常只能在其拟合的特定场景中优化其表示形式，而不会利用先前看到的图像中的任何先前信息。在本文中，我们建议使用在大型现有数据集中训练的模型提取的功能，以提高细分性能。我们通过体积渲染特征图和从每个输入图像提取的特征进行监督，将此特征表示形式烘烤到神经辐射场（NERF）中。我们表明，通过将此表示形式烘烤到NERF中，我们可以使后续的分类任务变得更加容易。我们的实验表明，与在各种场景中现有方法相比，我们的方法具有更高的分割精度，语义注释较少。
Methods have recently been proposed that densely segment 3D volumes into classes using only color images and expert supervision in the form of sparse semantically annotated pixels. While impressive, these methods still require a relatively large amount of supervision and segmenting an object can take several minutes in practice. Such systems typically only optimize their representation on the particular scene they are fitting, without leveraging any prior information from previously seen images. In this paper, we propose to use features extracted with models trained on large existing datasets to improve segmentation performance. We bake this feature representation into a Neural Radiance Field (NeRF) by volumetrically rendering feature maps and supervising on features extracted from each input image. We show that by baking this representation into the NeRF, we make the subsequent classification task much easier. Our experiments show that our method achieves higher segmentation accuracy with fewer semantic annotations than existing methods over a wide range of scenes.
