提升视觉语言模型对物体数量的感知能力，实现零样本通用计数
Text-promptable Object Counting via Quantity Awareness Enhancement
- 设计数量导向文本提示与视觉-文本数量对齐损失，增强模型数量意识
- 提出双流自适应计数解码器，融合Transformer与CNN优势，提升密度图预测精度
- 适用于跨类别、零样本场景，适合需要高泛化能力的计数应用
大型视觉语言模型在文本提示物体计数任务中取得显著进展，但现有方法仅依赖类别提示，难以准确区分物体数量。为此，我们提出QUANet，引入面向数量的文本提示，并设计视觉-文本数量对齐损失，增强模型的数量感知能力。同时，提出双流自适应计数解码器，包含Transformer流、CNN流及一系列Transformer-to-CNN增强适配器（T2C-adapters），实现两流间有效知识传递与融合。最后，通过跨流数量排序损失优化两流预测结果的排序关系。在FSC-147、CARPK、PUCPR+和ShanghaiTech等标准基准上的大量实验表明，该模型具备强大的零样本无类别计数泛化能力。代码已公开于https://github.com/viscom-tongji/QUANet。
原文摘要 · Abstract (English)
Recent advances in large vision-language models (VLMs) have shown remarkable progress in solving the text-promptable object counting problem. Representative methods typically specify text prompts with object category information in images. This however is insufficient for training the model to accurately distinguish the number of objects in the counting task. To this end, we propose QUANet, which introduces novel quantity-oriented text prompts with a vision-text quantity alignment loss to enhance the model's quantity awareness. Moreover, we propose a dual-stream adaptive counting decoder consisting of a Transformer stream, a CNN stream, and a number of Transformer-to-CNN enhancement adapters (T2C-adapters) for density map prediction. The T2C-adapters facilitate the effective knowledge communication and aggregation between the Transformer and CNN streams. A cross-stream quantity ranking loss is proposed in the end to optimize the ranking orders of predictions from the two streams. Extensive experiments on standard benchmarks such as FSC-147, CARPK, PUCPR+, and ShanghaiTech demonstrate our model's strong generalizability for zero-shot class-agnostic counting. Code is available at https://github.com/viscom-tongji/QUANet
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。