论文标题
学习将结构知识纳入图像介入
Learning to Incorporate Structure Knowledge for Image Inpainting
论文作者
论文摘要
本文开发了一个多任务学习框架，该框架试图合并图像结构知识以帮助图像介入，这在以前的作品中尚未很好地探索。主要思想是训练共享的生成器，以同时完成损坏的图像和相应的结构---边缘和梯度，从而暗中鼓励发电机在插入时利用相关的结构知识。同时，我们还引入了一种结构嵌入方案，将学习的结构特征明确嵌入到覆盖过程中，从而为图像完成提供了可能的前提。具体而言，提出了一种新型的金字塔结构损失来监督结构学习和嵌入。此外，开发了一种注意机制，以进一步利用图像中的复发结构和模式，以完善生成的结构和内容。通过多任务学习，除了注意之外，结构嵌入了，我们的框架利用了结构知识，并在定量和定性上优于基准数据集中的几种最新方法。
This paper develops a multi-task learning framework that attempts to incorporate the image structure knowledge to assist image inpainting, which is not well explored in previous works. The primary idea is to train a shared generator to simultaneously complete the corrupted image and corresponding structures --- edge and gradient, thus implicitly encouraging the generator to exploit relevant structure knowledge while inpainting. In the meantime, we also introduce a structure embedding scheme to explicitly embed the learned structure features into the inpainting process, thus to provide possible preconditions for image completion. Specifically, a novel pyramid structure loss is proposed to supervise structure learning and embedding. Moreover, an attention mechanism is developed to further exploit the recurrent structures and patterns in the image to refine the generated structures and contents. Through multi-task learning, structure embedding besides with attention, our framework takes advantage of the structure knowledge and outperforms several state-of-the-art methods on benchmark datasets quantitatively and qualitatively.
