论文标题
深入研究解码器以进行视频字幕
Delving Deeper into the Decoder for Video Captioning
论文作者
论文摘要
视频字幕是一项高级多模式任务，旨在用自然语言句子描述视频剪辑。编码器框架框架是近年来最受欢迎的该任务的最受欢迎的范式。但是，视频字幕模型的解码器中存在一些问题。我们对解码器进行了彻底的调查，并采用了三种技术来提高模型的性能。首先，将变分辍学和层归一化的组合嵌入了复发单元中，以减轻过度拟合的问题。其次，提出了一种新的在线方法来评估验证集上模型的性能，以便选择最佳的测试检查点。最后，提出了一种称为专业学习的新培训策略，该策略利用字幕模型的优势并绕过了其弱点。在Microsoft研究视频描述语料库（MSVD）和文本（MSR-VTT）数据集的实验中证明了这一点，我们的模型已通过BLEU，CIDER，METEOR和ROUGE-L指标获得了最佳结果，并在MSVD上获得了2.5％的高度增长，并在MSVD上获得了3.5％的高度增长。
Video captioning is an advanced multi-modal task which aims to describe a video clip using a natural language sentence. The encoder-decoder framework is the most popular paradigm for this task in recent years. However, there exist some problems in the decoder of a video captioning model. We make a thorough investigation into the decoder and adopt three techniques to improve the performance of the model. First of all, a combination of variational dropout and layer normalization is embedded into a recurrent unit to alleviate the problem of overfitting. Secondly, a new online method is proposed to evaluate the performance of a model on a validation set so as to select the best checkpoint for testing. Finally, a new training strategy called professional learning is proposed which uses the strengths of a captioning model and bypasses its weaknesses. It is demonstrated in the experiments on Microsoft Research Video Description Corpus (MSVD) and MSR-Video to Text (MSR-VTT) datasets that our model has achieved the best results evaluated by BLEU, CIDEr, METEOR and ROUGE-L metrics with significant gains of up to 18% on MSVD and 3.5% on MSR-VTT compared with the previous state-of-the-art models.
