论文标题
通过无提案掩盖的半监督时间动作检测
Semi-Supervised Temporal Action Detection with Proposal-Free Masking
论文作者
论文摘要
现有的时间动作检测（TAD）方法依赖于带有细分级注释的大量培训数据。因此，收集和注释这样的训练集非常昂贵且不可计入。半监督TAD（SS-TAD）通过利用无标记的视频大规模可用的视频来减轻此问题。但是，SS-Tad也比有监督的TAD更具挑战性的问题，因此研究得多。先前的SS-TAD方法直接结合了现有的基于提案的TAD方法和SSL方法。由于它们的顺序定位（例如，提案生成）和分类设计，它们很容易出现误差传播。为了克服这一局限性，在这项工作中，我们提出了一种基于无建议的时间面膜（点），具有平行定位（掩码生成）和分类体系结构的新型半监督时间动作检测模型。这种新颖的设计通过切断介于两者之间的错误传播途径来有效地消除了定位和分类之间的依赖性。我们进一步介绍了用于预测细化的分类和本地化之间的相互作用机制，以及用于自我监督模型预训练的新借口任务。对两个标准基准测试的广泛实验表明，我们的现场表现优于最先进的替代方案，通常会大幅度差。 pytorch实现现场可从https://github.com/sauradip/spot获得
Existing temporal action detection (TAD) methods rely on a large number of training data with segment-level annotations. Collecting and annotating such a training set is thus highly expensive and unscalable. Semi-supervised TAD (SS-TAD) alleviates this problem by leveraging unlabeled videos freely available at scale. However, SS-TAD is also a much more challenging problem than supervised TAD, and consequently much under-studied. Prior SS-TAD methods directly combine an existing proposal-based TAD method and a SSL method. Due to their sequential localization (e.g, proposal generation) and classification design, they are prone to proposal error propagation. To overcome this limitation, in this work we propose a novel Semi-supervised Temporal action detection model based on PropOsal-free Temporal mask (SPOT) with a parallel localization (mask generation) and classification architecture. Such a novel design effectively eliminates the dependence between localization and classification by cutting off the route for error propagation in-between. We further introduce an interaction mechanism between classification and localization for prediction refinement, and a new pretext task for self-supervised model pre-training. Extensive experiments on two standard benchmarks show that our SPOT outperforms state-of-the-art alternatives, often by a large margin. The PyTorch implementation of SPOT is available at https://github.com/sauradip/SPOT
