论文标题
通过重新组装姿势视频的任意对象的跨认同运动转移
Cross-Identity Motion Transfer for Arbitrary Objects through Pose-Attentive Video Reassembling
论文作者
论文摘要
我们提出了一个基于注意力的网络，用于在任意对象之间转移运动。给定源图像和驾驶视频，我们的网络根据驾驶视频中的动作在源图像中对主题进行动画动画。在我们的注意机制中，计算源和驾驶图像中学习的关键点之间的密集相似性，以从源图像中检索外观信息。从基于翘曲的模型中采用不同的方法，我们的基于注意力的模型具有多个优点。通过从源内容物中重新组装非本地搜索的零件，我们的方法可以产生更现实的输出。此外，我们的系统可以利用对源外观的多次观察（例如面部和面部的侧面），以使结果更加准确。为了减少自我监督学习的训练测试差异，还引入了一种新颖的跨认同培训计划。通过培训方案，我们的网络经过培训，可以在不同受试者之间转移运动，例如在实际测试方案中。实验结果验证了我们的方法在各种对象域中产生视觉上令人愉悦的结果，与以前的作品相比显示出更好的性能。
We propose an attention-based networks for transferring motions between arbitrary objects. Given a source image(s) and a driving video, our networks animate the subject in the source images according to the motion in the driving video. In our attention mechanism, dense similarities between the learned keypoints in the source and the driving images are computed in order to retrieve the appearance information from the source images. Taking a different approach from the well-studied warping based models, our attention-based model has several advantages. By reassembling non-locally searched pieces from the source contents, our approach can produce more realistic outputs. Furthermore, our system can make use of multiple observations of the source appearance (e.g. front and sides of faces) to make the results more accurate. To reduce the training-testing discrepancy of the self-supervised learning, a novel cross-identity training scheme is additionally introduced. With the training scheme, our networks is trained to transfer motions between different subjects, as in the real testing scenario. Experimental results validate that our method produces visually pleasing results in various object domains, showing better performances compared to previous works.
