用多模态查询和双流注意力提升交通场景目标检测精度与效率
Dual-Stream Attention with Multi-Modal Queries for Object Detection in Transportation Applications
- 引入视觉-语言、位置嵌入和随机学习三类查询，实现动态适配
- 在四个基准上达到最高平均精度（AP）和召回率，尤其在遮挡场景表现优
- 适合需要高精度定位的智能交通系统，如自动驾驶感知模块
基于Transformer的目标检测器常因固定查询和密集注意力导致遮挡处理差、细粒度定位弱及计算效率低。本文提出DAMM（Dual-stream Attention with Multi-Modal queries）框架，通过查询自适应与结构化跨注意力机制提升精度与效率。DAMM融合三类查询：来自视觉-语言模型的外观查询、基于多边形嵌入的位置查询，以及用于覆盖全局场景的随机学习查询。此外，双流跨注意力模块分别优化语义与空间特征，显著提升复杂场景下的定位精度。在四个挑战性基准上评估，DAMM在平均精度（AP）和召回率方面均达到当前最优表现，验证了多模态查询自适应与双流注意力的有效性。代码已开源：GitHub。
原文摘要 · Abstract (English)
Transformer-based object detectors often struggle with occlusions, fine-grained localization, and computational inefficiency caused by fixed queries and dense attention. We propose DAMM, Dual-stream Attention with Multi-Modal queries, a novel framework introducing both query adaptation and structured cross-attention for improved accuracy and efficiency. DAMM capitalizes on three types of queries: appearance-based queries from vision-language models, positional queries using polygonal embeddings, and random learned queries for general scene coverage. Furthermore, a dual-stream cross-attention module separately refines semantic and spatial features, boosting localization precision in cluttered scenes. We evaluated DAMM on four challenging benchmarks, and it achieved state-of-the-art performance in average precision (AP) and recall, demonstrating the effectiveness of multi-modal query adaptation and dual-stream attention. Source code is at: \href{https://github.com/DET-LIP/DAMM}{GitHub}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。