arXiv:2606.05586cs.CVcs.MM2026-06

用强化学习动态组合CNN与ViT模块,提升遥感目标检测性能。

BMCR: Adaptive Backbone Module Composition via Reinforcement Learning for Remote Sensing Object Detection

论文配图:BMCR: Adaptive Backbone Module Composition via Reinforcement Learning for Remote Sensing Object Detection
图 1 · 摘自论文原文
  • 通过强化学习动态选择CNN和ViT的可复用模块,构建输入自适应路径。
  • 在DOTA-v1.0等数据集上达到79.31% mAP,比基线最高提升2.5个百分点。
  • 适合需要灵活适配复杂遥感图像的目标检测任务,尤其关注模型效率与精度平衡者。

在遥感目标检测中,卷积神经网络(CNN)擅长捕捉局部细节,视觉变换器(ViT)则更擅长全局上下文建模。然而,现有检测器通常依赖单一固定主干或手动设计的混合架构,无法自适应地利用两者在不同输入下的互补优势。为此,我们提出基于强化学习的主干模块组合方法(BMCR)。BMCR从现成的CNN与ViT主干中分解出可复用的功能模块,动态构建输入自适应的推理路径。为支持跨架构组合,我们构建了一个可扩展的模块工具箱,将典型CNN与ViT主干分解为可复用模块,并封装其结构、语义和计算元数据以实现兼容性组装。为弥合网格化CNN特征与基于令牌的ViT表示之间的差距,设计了轻量级基于最优传输(OT)的转换接口,确保分布感知对齐的同时保持空间一致性。主干组合过程被建模为序列决策问题,策略网络根据多尺度中间观测逐步选择任务相关模块。为稳定可复用模块与路由策略的联合优化,进一步提出自适应模块协同优化(AMCO)策略,协调模块更新、路由探索与奖励分配。在DOTA-v1.0、DOTA-v1.5和DIOR-R数据集上,BMCR分别取得79.31%、73.41%和71.86%的mAP,超越强静态与动态基线达2.5个百分点,同时保持高效性。

原文摘要 · Abstract (English)

In remote sensing object detection, Convolutional Neural Networks (CNNs) excel at capturing local details while Vision Transformers (ViTs) are better at global context modeling. However, existing detectors typically rely on a single fixed backbone or a manually designed hybrid architecture, and thus fail to adaptively exploit these complementary strengths across inputs of diverse complexity. To address this limitation, we propose Backbone Module Composition via Reinforcement Learning (BMCR). BMCR dynamically assembles input-adaptive inference paths from reusable modules decomposed from off-the-shelf CNN and ViT backbones. To enable such cross-family composition, we first construct an extensible module toolbox. Specifically, we decompose representative CNN and ViT backbones into reusable functional modules and encapsulate each module with explicit structural, semantic, and computational metadata for compatibility-aware assembly. To bridge the gap between grid-based CNN features and token-based ViT representations, we design a lightweight Optimal Transport (OT) based transition interface that ensures distribution-aware alignment while respecting spatial consistency. The backbone composition process is then formulated as a sequential decision problem, in which a policy network progressively selects task-relevant modules according to intermediate multi-scale observations. To stabilize the joint optimization of reusable modules and the routing policy, we further develop an Adaptive Module Cooperative Optimization (AMCO) strategy that coordinates module updating, routing exploration, and reward assignment during training. On DOTA-v1.0, DOTA-v1.5 and DIOR-R, BMCR achieves 79.31\%, 73.41\% and 71.86\% mAP, respectively, surpassing strong static and dynamic baselines by up to 2.5 points while maintaining competitive efficiency.

遥感检测动态主干强化学习ViT-CNN融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。