论文标题
在野外重建手动相互作用
Reconstructing Hand-Object Interactions in the Wild
论文作者
论文摘要
在这项工作中，我们探讨了在野外重建手动对象的相互作用。这个问题的核心挑战是缺乏适当的3D标记数据。为了克服这个问题，我们提出了一个基于优化的程序，该程序不需要直接的3D监督。我们采用的一般策略是利用所有可用的相关数据（2D边界框，2D Hand Kepoints，2D实例掩码，3D对象模型，3D对象模型，LAB内MOCAP），以提供3D重建的约束。与其单独优化手和对象，我们可以共同优化它们，这使我们能够基于手动触点，碰撞和遮挡施加其他约束。我们的方法对来自Epic Kitchens和100天的Hands Dataset的充满挑战的内在数据产生了引人注目的重建，这些数据遍布一系列对象类别。从数量上讲，我们证明我们的方法与可用地面真相3D注释的实验室环境中的现有方法相比。
In this work we explore reconstructing hand-object interactions in the wild. The core challenge of this problem is the lack of appropriate 3D labeled data. To overcome this issue, we propose an optimization-based procedure which does not require direct 3D supervision. The general strategy we adopt is to exploit all available related data (2D bounding boxes, 2D hand keypoints, 2D instance masks, 3D object models, 3D in-the-lab MoCap) to provide constraints for the 3D reconstruction. Rather than optimizing the hand and object individually, we optimize them jointly which allows us to impose additional constraints based on hand-object contact, collision, and occlusion. Our method produces compelling reconstructions on the challenging in-the-wild data from the EPIC Kitchens and the 100 Days of Hands datasets, across a range of object categories. Quantitatively, we demonstrate that our approach compares favorably to existing approaches in the lab settings where ground truth 3D annotations are available.
