arXiv:2606.17561cs.CV2026-06

实时文本引导计数模型,精度高且速度超快。

RT-Counter: Real-Time Text-Guided Open-Vocabulary Object Counting

论文配图:RT-Counter: Real-Time Text-Guided Open-Vocabulary Object Counting
图 1 · 摘自论文原文
  • 用视觉原型文本化模块融合视觉与语言特征
  • 在FSC147上达13.30的低平均绝对误差
  • 每秒处理112帧,比现有方法快7.4倍

文本引导的开放词汇物体计数(TOOC)旨在统计由自然语言描述指定类别的物体数量。尽管视觉语言预训练模型已在该任务中取得进展,但在细粒度空间理解与实时推理方面仍存在挑战。本文提出实时计数框架RT-Counter,兼顾高精度与高效计算。其设计了视觉原型文本化(VPT)模块,将学习到的视觉特征映射至文本特征空间,生成包含抽象信息与细节原型信息的特征,增强视觉-语言模型的计数能力。同时引入编织注意力(Weaformer)层,采用混合注意力机制高效融合局部与全局视觉特征,在极低计算成本下保持强表达能力。在三个公开数据集上的实验表明,RT-Counter成功打破计数任务中的精度-速度权衡:在FSC147上实现13.30的平均绝对误差(MAE),达到112.48 FPS的推理速度,较现有最优方法快7.4倍,参数量减少逾4倍。代码已开源。

原文摘要 · Abstract (English)

Text-guided open-vocabulary object counting (TOOC) aims to count objects belonging to the categories specified by natural language descriptions. Although vision-language pre-trained models have been successful applied to TOOC tasks, they still struggle with fine-grained spatial understanding and real-time inference requirements in counting scenarios. To address these limitations, this paper proposes a real-time TOOC framework, called the Real-Time Counter (RT-Counter), that achieves not only good counting accuracy but also high computational efficiency. RT-Counter designs a novel Visual Prototype Textualization (VPT) module that can project learned visual features into a text feature space and then generate features containing the abstract information that is hard to capture with visual prototypes and the detailed prototype information that is difficult to describe in text, enhancing the object-level visual-language model's counting capabilities. Additionally, RT-Counter incorporates our Weaving Transformer (Weaformer) layers, maintaining high descriptive power at a fraction of the computational cost. The Weaformer layer adopts a novel hybrid attention mechanism that can efficiently weave together local and global visual features. Extensive experiments on three public datasets show that RT-Counter successfully breaks the accuracy-speed trade-off in TOOC. While achieving a competitive MAE of 13.30 on FSC147, RT-Counter operates at 112.48 FPS, making it 7.4x faster and over 4$\times$ more parameter-efficient than the existing leading methods in TOOC. Our work aims at balancing high accuracy and real-time performance in TOOC. Code is available at: https://github.com/Jason-Mar1/RT-Counter.

物体计数文本引导实时推理视觉语言

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。