论文标题
KOVIS：基于按键的视觉伺服伺服击形，以零射击SIM到现实转移用于机器人操纵
KOVIS: Keypoint-based Visual Servoing with Zero-Shot Sim-to-Real Transfer for Robotics Manipulation
论文作者
论文摘要
我们提出了Kovis，这是一种基于学习的新型，无校准的视觉伺服方法，用于使用眼睛的立体声相机系统进行精细的机器人操纵任务。我们仅在模拟环境中训练深度神经网络；训练有素的模型可直接用于现实世界的视觉伺服任务。 Kovis由两个网络组成。第一个关键点网络使用自动编码器从图像中学习关键点表示。然后，Visual Servoing网络根据从相机图像中提取的关键点学习运动。这两个网络通过无需手动数据标记而在模拟环境中端到端训练。在使用数据增强，域随机化和对抗性示例进行培训之后，我们能够实现零击的SIM到现实传输转移到现实世界的机器人操纵任务。我们证明了该方法在模拟环境和实验实验中的有效性，包括不同的机器人操纵任务，包括抓握，插入4mm间隙和M13螺丝插入。该演示视频可从http://youtu.be/gfbjbr2tdza获得
We present KOVIS, a novel learning-based, calibration-free visual servoing method for fine robotic manipulation tasks with eye-in-hand stereo camera system. We train the deep neural network only in the simulated environment; and the trained model could be directly used for real-world visual servoing tasks. KOVIS consists of two networks. The first keypoint network learns the keypoint representation from the image using with an autoencoder. Then the visual servoing network learns the motion based on keypoints extracted from the camera image. The two networks are trained end-to-end in the simulated environment by self-supervised learning without manual data labeling. After training with data augmentation, domain randomization, and adversarial examples, we are able to achieve zero-shot sim-to-real transfer to real-world robotic manipulation tasks. We demonstrate the effectiveness of the proposed method in both simulated environment and real-world experiment with different robotic manipulation tasks, including grasping, peg-in-hole insertion with 4mm clearance, and M13 screw insertion. The demo video is available at http://youtu.be/gfBJBR2tDzA
