arXiv:2507.19807cs.CV2025-07

提出灵活检测新框架,解决注意力冲突与查询模糊问题

DS-Det: Single-Query Paradigm and Attention Disentangled Learning for Flexible Object Detection

  • 采用单查询范式,让检测数量可变,突破固定查询限制
  • 分离交叉注意力与自注意力,提升效率并解决查询混淆
  • 适配小目标检测,适合追求高效灵活检测的开发者

主流Transformer检测器依赖基于注意力的查询学习,但现有解码器查询类型(如内容查询、位置查询)的作用仍不明确。传统方法使用固定数量的预定义查询,限制了灵活性。我们发现,这种固定查询的学习受自注意力(查询到查询)与交叉注意力(查询到编码器)之间循环对立作用(ROT)的影响,降低解码器效率。此外,在共享权重解码层中同时处理一对一和一对多标签分配时,会引发“查询歧义”,违背DETR的一一匹配原则。为此,我们提出DS-Det,一种可灵活检测任意数量物体的高效检测器。具体地,我们引入统一的单查询范式,将固定查询转化为可变形式;通过注意力解耦学习设计简化解码框架:用交叉注意力定位候选框(一对多),用自注意力去重预测(一对一),直接解决“查询歧义”与“ROT”问题,提升效率。进一步提出统一的PoCoo损失,利用框大小先验,优先优化难样本(如小目标)的查询学习。在五个不同主干模型上,于COCO2017和WiderPerson数据集上的大量实验验证了DS-Det的通用性与优越性。

原文摘要 · Abstract (English)

Popular transformer detectors have achieved promising performance through query-based learning using attention mechanisms. However, the roles of existing decoder query types (e.g., content query and positional query) are still underexplored. These queries are generally predefined with a fixed number (fixed-query), which limits their flexibility. We find that the learning of these fixed-query is impaired by Recurrent Opposing inTeractions (ROT) between two attention operations: Self-Attention (query-to-query) and Cross-Attention (query-to-encoder), thereby degrading decoder efficiency. Furthermore, "query ambiguity" arises when shared-weight decoder layers are processed with both one-to-one and one-to-many label assignments during training, violating DETR's one-to-one matching principle. To address these challenges, we propose DS-Det, a more efficient detector capable of detecting a flexible number of objects in images. Specifically, we reformulate and introduce a new unified Single-Query paradigm for decoder modeling, transforming the fixed-query into flexible. Furthermore, we propose a simplified decoder framework through attention disentangled learning: locating boxes with Cross-Attention (one-to-many process), deduplicating predictions with Self-Attention (one-to-one process), addressing "query ambiguity" and "ROT" issues directly, and enhancing decoder efficiency. We further introduce a unified PoCoo loss that leverages box size priors to prioritize query learning on hard samples such as small objects. Extensive experiments across five different backbone models on COCO2017 and WiderPerson datasets demonstrate the general effectiveness and superiority of DS-Det. The source codes are available at https://github.com/Med-Process/DS-Det/.

目标检测Transformer灵活检测注意力机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。