论文标题
AUG-NERF：具有三重层面的增强的训练更强的神经辐射场
Aug-NeRF: Training Stronger Neural Radiance Fields with Triple-Level Physically-Grounded Augmentations
论文作者
论文摘要
神经辐射场（NERF）通过通过地面真相监督差异渲染多视图图像来回归神经参数化的场景。但是，当插值新颖的观点时，NERF通常会产生不一致且视觉上不平滑的几何结果，我们认为这是可见和看不见的观点之间的概括差距。卷积神经网络的最新进展表明，随机或学到的先进的鲁棒数据增强有望增强分布和分布外的概括。受到此启发，我们提出了增强的NERF（Aug-nerf），这首先将强大的数据增强功能带入正规化NERF培训。特别是，我们的建议学会了将最坏情况扰动无缝融合到NERF管道的三个不同级别的物理基础，包括（1）输入坐标，以模拟图像捕获时的不精确摄像机参数； （2）中间特征，以平滑固有的特征歧管； （3）预先渲染的输出，以说明多视图图像监督中的潜在降解因子。广泛的结果表明，Aug-nerf在新型视图合成（高达1.5dB PSNR增益）和基础几何重建中有效地提高了NERF性能。此外，得益于三级增强的隐含平稳先验，Aug-nerf甚至可以从严重损坏的图像中恢复场景，这是一个富有挑战性的环境，以前曾经不受欢迎。我们的代码可在https://github.com/vita-group/aug-nerf中找到。
Neural Radiance Field (NeRF) regresses a neural parameterized scene by differentially rendering multi-view images with ground-truth supervision. However, when interpolating novel views, NeRF often yields inconsistent and visually non-smooth geometric results, which we consider as a generalization gap between seen and unseen views. Recent advances in convolutional neural networks have demonstrated the promise of advanced robust data augmentations, either random or learned, in enhancing both in-distribution and out-of-distribution generalization. Inspired by that, we propose Augmented NeRF (Aug-NeRF), which for the first time brings the power of robust data augmentations into regularizing the NeRF training. Particularly, our proposal learns to seamlessly blend worst-case perturbations into three distinct levels of the NeRF pipeline with physical grounds, including (1) the input coordinates, to simulate imprecise camera parameters at image capture; (2) intermediate features, to smoothen the intrinsic feature manifold; and (3) pre-rendering output, to account for the potential degradation factors in the multi-view image supervision. Extensive results demonstrate that Aug-NeRF effectively boosts NeRF performance in both novel view synthesis (up to 1.5dB PSNR gain) and underlying geometry reconstruction. Furthermore, thanks to the implicit smooth prior injected by the triple-level augmentations, Aug-NeRF can even recover scenes from heavily corrupted images, a highly challenging setting untackled before. Our codes are available in https://github.com/VITA-Group/Aug-NeRF.
