用扩散模型提升文本引导计数的敏感度，解决零样本计数难题。
T2ICount: Enhancing Cross-modal Understanding for Zero-Shot Counting
- 基于扩散模型构建框架，利用其精细视觉理解能力。
- 在FSC147上实现9.8%的相对提升，优于现有方法。
- 适合研究跨模态理解与零样本计数的学者参考。
零样本目标计数旨在根据文本描述统计任意类别物体的数量。现有方法通常依赖CLIP等视觉语言模型，但对文本提示的敏感度有限。本文提出T2ICount，一种基于扩散模型的框架，利用预训练扩散模型丰富的先验知识和细粒度视觉理解能力。尽管单步去噪保证了效率，但削弱了文本敏感性。为此，我们设计了分层语义修正模块，逐步优化文本-图像特征对齐；并引入表征区域一致性损失，通过去噪U-Net提取的交叉注意力图提供可靠监督信号。此外，我们发现现有基准主要关注图像中的多数物体，可能掩盖模型的文本敏感性。为此，我们贡献了一个重新标注的FSC147挑战子集，以更准确评估文本引导计数能力。大量实验表明，该方法在多个基准上均取得优异性能。代码已公开于https://github.com/cha15yq/T2ICount。
原文摘要 · Abstract (English)
Zero-shot object counting aims to count instances of arbitrary object categories specified by text descriptions. Existing methods typically rely on vision-language models like CLIP, but often exhibit limited sensitivity to text prompts. We present T2ICount, a diffusion-based framework that leverages rich prior knowledge and fine-grained visual understanding from pretrained diffusion models. While one-step denoising ensures efficiency, it leads to weakened text sensitivity. To address this challenge, we propose a Hierarchical Semantic Correction Module that progressively refines text-image feature alignment, and a Representational Regional Coherence Loss that provides reliable supervision signals by leveraging the cross-attention maps extracted from the denosing U-Net. Furthermore, we observe that current benchmarks mainly focus on majority objects in images, potentially masking models' text sensitivity. To address this, we contribute a challenging re-annotated subset of FSC147 for better evaluation of text-guided counting ability. Extensive experiments demonstrate that our method achieves superior performance across different benchmarks. Code is available at https://github.com/cha15yq/T2ICount.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。