论文标题
使用单色点投影深度高光谱重建
Deep Hyperspectral-Depth Reconstruction Using Single Color-Dot Projection
论文作者
论文摘要
深度重建和高光谱反射重建是计算机视觉和图像处理中的两个主动研究主题。通常，这两个主题已使用独立的成像设置分别研究了，并且没有现有的方法可以在不使用特殊硬件的情况下一镜头同时获得深度和光谱反射率。在本文中，我们建议使用现成的RGB摄像头和投影仪提出一种新型的单发性高光谱重建方法。我们的方法基于单个颜色点投影，该投影同时充当深度重建和空间变化的颜色照明的结构光，用于高光谱反射率重建。为了共同重构单个颜色点图像的深度和高光谱反射率，我们提出了一种新颖的端到端网络结构，有效地结合了几何色点模式损失和光度超光谱反射率损失。通过实验，我们证明了我们的高光谱重建方法优于现有的最新单光超光反射重建方法和深度重建方法的组合。
Depth reconstruction and hyperspectral reflectance reconstruction are two active research topics in computer vision and image processing. Conventionally, these two topics have been studied separately using independent imaging setups and there is no existing method which can acquire depth and spectral reflectance simultaneously in one shot without using special hardware. In this paper, we propose a novel single-shot hyperspectral-depth reconstruction method using an off-the-shelf RGB camera and projector. Our method is based on a single color-dot projection, which simultaneously acts as structured light for depth reconstruction and spatially-varying color illuminations for hyperspectral reflectance reconstruction. To jointly reconstruct the depth and the hyperspectral reflectance from a single color-dot image, we propose a novel end-to-end network architecture that effectively incorporates a geometric color-dot pattern loss and a photometric hyperspectral reflectance loss. Through the experiments, we demonstrate that our hyperspectral-depth reconstruction method outperforms the combination of an existing state-of-the-art single-shot hyperspectral reflectance reconstruction method and depth reconstruction method.
