让模型既能数你想数的，也能排除你不想要的物体。
CountGD++: Generalized Prompting for Open-World Counting
- 用文字或图像说明要排除的物体，突破传统只能指定目标的限制。
- 自动标注视觉示例，无需人工干预，提升效率与泛化能力。
- 支持自然与合成图像的示例输入，适合多场景开放世界计数任务。
现有方法在自动图像和视频中计数时，受限于目标物体的指定方式。尽管已有方法允许用户通过文本和视觉样例描述目标物体，但视觉样例需手动标注，且无法指定不计数的物体。为解决此问题，本文引入三项新能力：1）扩展提示以支持用文本和/或视觉样例描述应排除的物体；2）提出‘伪样例’（pseudo-exemplars）机制，在推理时自动标注视觉样例；3）将计数模型扩展至接受来自自然与合成外部图像的视觉样例。我们还构建了新型计数模型 CountGD++，作为视觉专家代理供大语言模型调用。这些改进显著提升了多数据集上的计数准确性、效率与泛化能力。代码已开源：https://github.com/niki-amini-naieni/CountGDPlusPlus。
原文摘要 · Abstract (English)
The flexibility and accuracy of methods for automatically counting objects in images and videos are limited by the way the object can be specified. While existing methods allow users to describe the target object with text and visual examples, the visual examples must be manually annotated inside the image, and there is no way to specify what not to count. To address these gaps, we introduce novel capabilities that expand how the target object can be specified. Specifically, we extend the prompt to enable what not to count to be described with text and/or visual examples, introduce the concept of `pseudo-exemplars' that automate the annotation of visual examples at inference, and extend counting models to accept visual examples from both natural and synthetic external images. We also use our new counting model, CountGD++, as a vision expert agent for an LLM. Together, these contributions expand the prompt flexibility of multi-modal open-world counting and lead to significant improvements in accuracy, efficiency, and generalization across multiple datasets. Code is available at https://github.com/niki-amini-naieni/CountGDPlusPlus.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。