论文标题
无人驾驶汽车的实时多模式语义融合，带有标签繁殖的标签
Real-Time Multi-Modal Semantic Fusion on Unmanned Aerial Vehicles with Label Propagation for Cross-Domain Adaptation
论文作者
论文摘要
配备多个互补传感器的无人飞行器（UAV）具有快速自动或遥控的语义场景分析（例如灾难检查）具有巨大的潜力。在这里，我们提出了一个无人机系统，用于实时语义推断和多种传感器方式的融合。 LIDAR扫描和RGB图像的语义分割，以及RGB和热图像上的对象检测，使用轻巧的CNN架构在UAV计算机上在线运行，并嵌入了推理加速器。我们遵循一种晚期的融合方法，其中来自多个传感器模式的语义信息增加了3D点云和图像分割掩码，同时还生成了同类语义图。语义图上的标签传播允许通过交叉模式和跨域监督进行传感器特定的适应。我们的系统提供了大约$ 9 Hz的增强语义图像和点云。我们在城市环境和灾难测试地点中评估了现实世界实验中的集成系统。
Unmanned aerial vehicles (UAVs) equipped with multiple complementary sensors have tremendous potential for fast autonomous or remote-controlled semantic scene analysis, e.g., for disaster examination. Here, we propose a UAV system for real-time semantic inference and fusion of multiple sensor modalities. Semantic segmentation of LiDAR scans and RGB images, as well as object detection on RGB and thermal images, run online onboard the UAV computer using lightweight CNN architectures and embedded inference accelerators. We follow a late fusion approach where semantic information from multiple sensor modalities augments 3D point clouds and image segmentation masks while also generating an allocentric semantic map. Label propagation on the semantic map allows for sensor-specific adaptation with cross-modality and cross-domain supervision. Our system provides augmented semantic images and point clouds with $\approx$ 9 Hz. We evaluate the integrated system in real-world experiments in an urban environment and at a disaster test site.
