论文标题
代表，比较和学习：类不足的计数的相似性感知框架
Represent, Compare, and Learn: A Similarity-Aware Framework for Class-Agnostic Counting
论文作者
论文摘要
类不足的计数（CAC）的目的是在查询图像中对所有实例进行计数，但几乎没有示例。标准管道是从示例中提取视觉特征，并与查询图像匹配以推断对象计数。该管道中的两个基本组件是特征表示和相似性度量。现有的方法要么采用预处理的网络来表示功能，要么学习新的方法，同时将幼稚的相似性度量与固定的内部产品应用。我们发现这种范式导致嘈杂的相似性匹配，从而损害了计数性能。在这项工作中，我们提出了一个相似性感知的CAC框架，该框架共同学习表示和相似性指标。我们首先使用称为双线性匹配网络（BMNET）的天真基线实例化框架，其关键组件是可学习的双线性相似性度量。为了进一步体现我们的框架的核心，我们将BMNET扩展到BMNET+，该BMNET+从三个方面进行了模型：1）通过它们的自相似性来代表实例，以增强针对类内部变化的特征鲁棒性； 2）将相似性动态比较与每个示例的关键模式进行比较； 3）从监督信号中学习，对匹配结果施加明确的限制。最近在CAC数据集FSC147上进行的广泛实验表明，我们的模型的表现明显优于最先进的CAC方法。此外，我们还验证了计数数据集腕带上BMNET和BMNET+的跨数据库一般性。代码在tiny.one/bmnet上
Class-agnostic counting (CAC) aims to count all instances in a query image given few exemplars. A standard pipeline is to extract visual features from exemplars and match them with query images to infer object counts. Two essential components in this pipeline are feature representation and similarity metric. Existing methods either adopt a pretrained network to represent features or learn a new one, while applying a naive similarity metric with fixed inner product. We find this paradigm leads to noisy similarity matching and hence harms counting performance. In this work, we propose a similarity-aware CAC framework that jointly learns representation and similarity metric. We first instantiate our framework with a naive baseline called Bilinear Matching Network (BMNet), whose key component is a learnable bilinear similarity metric. To further embody the core of our framework, we extend BMNet to BMNet+ that models similarity from three aspects: 1) representing the instances via their self-similarity to enhance feature robustness against intra-class variations; 2) comparing the similarity dynamically to focus on the key patterns of each exemplar; 3) learning from a supervision signal to impose explicit constraints on matching results. Extensive experiments on a recent CAC dataset FSC147 show that our models significantly outperform state-of-the-art CAC approaches. In addition, we also validate the cross-dataset generality of BMNet and BMNet+ on a car counting dataset CARPK. Code is at tiny.one/BMNet
