DA-Mamba用Mamba+注意力机制实现跨域目标检测,兼顾效率与性能。
DA-Mamba: Domain Adaptive Hybrid Mamba-Transformer Based One-Stage Object Detection
- 融合Mamba的线性计算与注意力机制,捕捉长程依赖。
- 在多个域适应数据集上达到领先性能,提升检测精度。
- 适合需要高效跨域检测的应用场景,如自动驾驶。
基于2D CNN的域适应方法因感受野有限,难以处理空间分布显著变化的目标域。虽然基于Transformer的方法通过自注意力机制能更好建模远距离关系,但其二次复杂度限制了在目标检测中的实际应用。受Mamba架构全局建模能力与线性计算复杂度启发,我们提出首个基于Mamba的域自适应单阶段目标检测模型DA-Mamba。具体地,将Mamba的状态空间建模与注意力机制结合,利用域自适应的空间与通道扫描提取可迁移特征,同时通过交叉注意力模块生成长程混合域空间特征,实现鲁棒的软对齐。此外,针对混合架构引入的特征噪声,提出基于熵的知识蒸馏框架,结合边缘ReLU,自适应抑制无关激活并统一源/目标域不确定性。最后,为防止交叉注意力生成混合特征导致过拟合,设计熵驱动的门控注意力与随机扰动机制,同步优化目标特征并增强泛化能力。
原文摘要 · Abstract (English)
Recent 2D CNN-based domain adaptation approaches struggle with long-range dependencies due to limited receptive fields, making it difficult to adapt to target domains with significant spatial distribution changes. While transformer-based domain adaptation methods better capture distant relationships through self-attention mechanisms that facilitate more effective cross-domain feature alignment, their quadratic computational complexity makes practical deployment challenging for object detection tasks across diverse domains. Inspired by the global modeling and linear computation complexity of the Mamba architecture, we present the first domain-adaptive Mamba-based one-stage object detection model, termed DA-Mamba. Specifically, we combine Mamba's efficient state-space modeling with attention mechanisms to address domain-specific spatial and channel-wise variations. Our design leverages domain-adaptive spatial and channel-wise scanning within the Mamba block to extract highly transferable representations for efficient sequential processing, while cross-attention modules generate long-range, mixed-domain spatial features to enable robust soft alignment across domains. Besides, motivated by the observation that hybrid architectures introduce feature noise in domain adaptation tasks, we propose an entropy-based knowledge distillation framework with margin ReLU, which adaptively refines multi-level representations by suppressing irrelevant activations and aligning uncertainty across source and target domains. Finally, to prevent overfitting caused by the mixed-up features generated through cross-attention mechanisms, we propose entropy-driven gating attention with random perturbations that simultaneously refine target features and enhance model generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。