论文标题
明确的图像字幕编辑
Explicit Image Caption Editing
论文作者
论文摘要
给定图像和参考字幕，图像标题编辑任务旨在纠正未对准错误并生成精致的字幕。但是，所有现有的字幕编辑作品都是隐式模型，即它们直接产生精致的字幕而无需与参考标题明确连接。在本文中，我们介绍了一项新任务：显式字幕编辑（ECE）。 ECE模型明确生成了一系列编辑操作，此编辑操作序列可以将参考字幕转换为精制的字幕。与隐式编辑相比，ECE具有多个优点：1）可解释：它可以追踪整个编辑路径。 2）编辑有效：它只需要修改几个单词。 3）类似人类：它类似于人类执行字幕编辑的方式，并试图保持原始句子结构。为了解决这项新任务，我们提出了第一个ECE模型：Tiger。 Tiger是一种非自动回形变压器的模型，由三个模块组成：Tagger_del，Tagger_Add和Inserster。具体来说，Tagger_del决定是否应该保留每个单词，Tagger_add决定添加新单词的位置，而Inserter预测了添加的特定单词。为了进一步促进ECE研究，我们分别重新组织了两个现有数据集，分别为Coco-EE和FlickR30K-EE，提出了两个新的ECE基准。两个基准上的大量消融都证明了老虎的有效性。
Given an image and a reference caption, the image caption editing task aims to correct the misalignment errors and generate a refined caption. However, all existing caption editing works are implicit models, ie, they directly produce the refined captions without explicit connections to the reference captions. In this paper, we introduce a new task: Explicit Caption Editing (ECE). ECE models explicitly generate a sequence of edit operations, and this edit operation sequence can translate the reference caption into a refined one. Compared to the implicit editing, ECE has multiple advantages: 1) Explainable: it can trace the whole editing path. 2) Editing Efficient: it only needs to modify a few words. 3) Human-like: it resembles the way that humans perform caption editing, and tries to keep original sentence structures. To solve this new task, we propose the first ECE model: TIger. TIger is a non-autoregressive transformer-based model, consisting of three modules: Tagger_del, Tagger_add, and Inserter. Specifically, Tagger_del decides whether each word should be preserved or not, Tagger_add decides where to add new words, and Inserter predicts the specific word for adding. To further facilitate ECE research, we propose two new ECE benchmarks by re-organizing two existing datasets, dubbed COCO-EE and Flickr30K-EE, respectively. Extensive ablations on both two benchmarks have demonstrated the effectiveness of TIger.
