通过多路径训练提升检测Transformer的性能，兼顾一对一与一对多预测。
Mr. DETR++: Instructive Multi-Route Training for Detection Transformers with Mixture-of-Experts
- 设计双辅助路径，分别用指导性自注意力和路由感知MoE实现一对多预测。
- 在多个检测基准上实现稳定提升，实例分割与全景分割也有效。
- 方法灵活易适配，推理时丢弃辅助路径，不影响效率。
现有方法通过引入辅助的一对多分配来增强检测Transformer的训练。本文将模型视为多任务框架，同时进行一对一和一对多预测。我们研究了Transformer解码器中各组件（自注意力、交叉注意力、前馈网络）在两种训练目标下的作用。实验表明，解码器中的任意独立组件均可同时有效学习两种目标，即使其他组件共享。基于此发现，我们提出多路径训练机制：主路径用于一对一预测，两条辅助路径用于一对多预测。第一辅助路径引入新颖的指导性自注意力机制，动态引导对象查询完成一对多预测；第二辅助路径采用路由感知的混合专家（MoE），促进知识共享并缓解路径间冲突。此外，我们在编码器低尺度特征中应用MoE，优化效率与效果平衡。辅助路径仅用于训练，推理时移除。我们在多种检测基线及实例分割、全景分割任务上进行广泛实验，结果一致显示性能提升。方法高度灵活，可拓展至其他任务。
原文摘要 · Abstract (English)
Existing methods enhance the training of detection transformers by incorporating an auxiliary one-to-many assignment. In this work, we treat the model as a multi-task framework, simultaneously performing one-to-one and one-to-many predictions. We investigate the roles of each component in the transformer decoder across these two training targets, including self-attention, cross-attention, and feed-forward network. Our empirical results demonstrate that any independent component in the decoder can effectively learn both targets simultaneously, even when other components are shared. This finding leads us to propose a multi-route training mechanism, featuring a primary route for one-to-one prediction and two auxiliary training routes for one-to-many prediction. We propose a novel instructive self-attention mechanism, integrated into the first auxiliary route, which dynamically and flexibly guides object queries for one-to-many prediction. For the second auxiliary route, we introduce a route-aware Mixture-of-Experts (MoE) to facilitate knowledge sharing while mitigating potential conflicts between routes. Additionally, we apply an MoE to low-scale features in the encoder, optimizing the balance between efficiency and effectiveness. The auxiliary routes are discarded during inference. We conduct extensive experiments across various object detection baselines, achieving consistent improvements as demonstrated in Fig. 1. Our method is highly flexible and can be readily adapted to other tasks. To demonstrate its versatility, we conduct experiments on both instance segmentation and panoptic segmentation, further validating its effectiveness. Project page: https://visual-ai.github.io/mrdetr/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。