论文标题
基于零拍的图像检索的渐进域无关的特征分解网络
Progressive Domain-Independent Feature Decomposition Network for Zero-Shot Sketch-Based Image Retrieval
论文作者
论文摘要
基于零拍的草图图像检索(ZS-SBIR)是在零拍摄方案下搜索自然手法草图的特定跨模式检索任务。大多数现有方法通过同时将视觉特征和语义监督投影到低维的公共空间以进行有效检索来解决此问题。但是,这种低维投影摧毁了原始语义空间中语义知识的完整性,因此从不同方式学习语义时,它无法很好地传输有用的知识。此外,域信息和语义信息与视觉特征纠缠在一起,这不利于交叉模式匹配,因为这会阻碍绘制草图和图像之间的域间隙的减少。在本文中,我们建议ZS-SBIR的渐进域独立于特征分解(PDFD)网络。具体而言,在对原始语义知识的监督下,PDFD将视觉特征分解为域特征和语义功能,然后将语义特征投射到公共空间中,作为ZS-SBIR的检索功能。渐进式投影策略保持了强大的语义监督。此外,为了确保捕获清洁和完整的语义信息的检索功能,引入了跨重建损失,以鼓励检索功能和域功能的任何组合都可以重建视觉功能。广泛的实验证明了我们的PDFD优于最先进的竞争对手。
Zero-shot sketch-based image retrieval (ZS-SBIR) is a specific cross-modal retrieval task for searching natural images given free-hand sketches under the zero-shot scenario. Most existing methods solve this problem by simultaneously projecting visual features and semantic supervision into a low-dimensional common space for efficient retrieval. However, such low-dimensional projection destroys the completeness of semantic knowledge in original semantic space, so that it is unable to transfer useful knowledge well when learning semantic from different modalities. Moreover, the domain information and semantic information are entangled in visual features, which is not conducive for cross-modal matching since it will hinder the reduction of domain gap between sketch and image. In this paper, we propose a Progressive Domain-independent Feature Decomposition (PDFD) network for ZS-SBIR. Specifically, with the supervision of original semantic knowledge, PDFD decomposes visual features into domain features and semantic ones, and then the semantic features are projected into common space as retrieval features for ZS-SBIR. The progressive projection strategy maintains strong semantic supervision. Besides, to guarantee the retrieval features to capture clean and complete semantic information, the cross-reconstruction loss is introduced to encourage that any combinations of retrieval features and domain features can reconstruct the visual features. Extensive experiments demonstrate the superiority of our PDFD over state-of-the-art competitors.