用参考图生成多样样本,让零样本计数更准。
RECOUNT: Reference-guided Counting with Synthetic Visual Exemplars

- 用扩散模型自动生成目标类别的视觉样例,弥补文本描述不足
- 在LookAlikes和PairTally上计数误差降低55%和21%
- 适合需要零样本精准计数且无标注数据的场景
文本引导的零样本物体计数器在空间定位上表现良好,但在新类别或细粒度类别上分类能力差:自然语言过于粗略,无法完整指定视觉身份,导致难以区分外观相似的干扰项。少样本计数器通过视觉样例解决此问题,但需每张图手动标注。为解决这一困境,我们提出RECOUNT,一个即插即用的图像引导零样本计数框架。不同于用文本提示指定类别,我们的核心思想是用单张离场参考图像视觉指定类别。然而,单一参考图像覆盖范围有限,跨场景可靠性差。因此,我们重新利用扩散模型作为自动对比数据生成引擎,将参考图像扩展为多样化样例库,提供文本无法提供的判别细节。RECOUNT保留任何冻结计数器的类别无关提议,并将分类任务交由独立视觉模块(冻结主干+轻量头部,在合成数据上训练)完成,该模块将每个提议与目标和干扰样例库匹配。应用于冻结计数器时,RECOUNT在两个基准上均达到最佳零样本准确率,相较最强先驱者,计数误差(MAE)在LookAlikes上降低55%,在PairTally上降低21%。
原文摘要 · Abstract (English)
Text-guided zero-shot object counters excel at spatial localization but categorize poorly on novel or fine-grained classes: natural language is too coarse to fully specify visual identity, so they fail to separate visually similar distractors. Few-shot counters sidestep this with visual exemplars, but require manual annotations on every image. To resolve this dilemma, we introduce RECOUNT, a plug-and-play framework for image-guided zero-shot counting. Rather than specify a category with a text prompt, our key insight is to specify it visually, from a single off-scene reference image. However, we find that a lone reference image provides narrow coverage of a category's appearance and is unreliable across diverse scenes. We therefore repurpose a diffusion model as an automated contrastive data engine that expands the reference into a diverse exemplar gallery, supplying the discriminative detail that text cannot. RECOUNT preserves the class-agnostic proposals of any frozen counter and offloads categorization to a separate visual module (a frozen backbone with a lightweight head trained on this synthetic data) that matches each proposal against the target and distractor galleries. Applied to a frozen counter, RECOUNT attains the best zero-shot accuracy on both benchmarks, cutting counting error (MAE) by 55% on LookAlikes and 21% on PairTally relative to the strongest prior zero-shot counter.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。