用空间注意力与自适应融合提升红外可见光人群计数精度
A Dual-Modulation Framework for RGB-T Crowd Counting via Spatially Modulated Attention and Adaptive Fusion
- 引入可学习的空间衰减掩码，抑制远距离无关区域注意力
- 动态门控机制按需选择可靠模态进行跨模态融合
- 在多个数据集上优于现有方法，适合复杂环境下的人群监控
准确的红外-可见光（RGB-T）人群计数对复杂环境下公共安全至关重要。尽管基于Transformer的方法擅长捕捉全局上下文，但其缺乏空间归纳偏置，导致注意力扩散至无关背景区域，影响人群定位精度。此外，有效弥合两种模态间的差异仍是主要挑战。为此，我们提出双调制框架，包含两个模块：空间调制注意力（SMA），通过可学习的空间衰减掩码惩罚远距离令牌间的注意力，防止关注点扩散至背景；自适应融合调制（AFM），采用动态门控机制，优先选择更可靠的模态以实现自适应跨模态融合。在多个RGB-T人群计数数据集上的大量实验表明，该方法显著优于现有工作。代码已开源：https://github.com/Cht2924/RGBT-Crowd-Counting。
原文摘要 · Abstract (English)
Accurate RGB-Thermal (RGB-T) crowd counting is crucial for public safety in challenging conditions. While recent Transformer-based methods excel at capturing global context, their inherent lack of spatial inductive bias causes attention to spread to irrelevant background regions, compromising crowd localization precision. Furthermore, effectively bridging the gap between these distinct modalities remains a major hurdle. To tackle this, we propose the Dual Modulation Framework, comprising two modules: Spatially Modulated Attention (SMA), which improves crowd localization by using a learnable Spatial Decay Mask to penalize attention between distant tokens and prevent focus from spreading to the background; and Adaptive Fusion Modulation (AFM), which implements a dynamic gating mechanism to prioritize the most reliable modality for adaptive cross-modal fusion. Extensive experiments on RGB-T crowd counting datasets demonstrate the superior performance of our method compared to previous works. Code available at https://github.com/Cht2924/RGBT-Crowd-Counting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。