用专家模型精准定位无人机与卫星图像中的同一物体。
SMGeo: Cross-View Object Geo-Localization with Grid-Level Mixture-of-Experts
- 基于网格级专家混合的Transformer架构，自适应融合多视角特征。
- 在IoU=0.25下准确率达87.51%，显著优于现有方法。
- 支持点击提示，可实时交互定位，适合地理信息应用。
跨视图物体地理定位旨在基于无人机图像精确定位大规模卫星影像中的同一物体。由于视角和尺度差异大，且背景干扰复杂，传统“检索-匹配”多阶段流程易产生累积误差。为此，我们提出SMGeo，一种可提示的端到端Transformer模型，支持点击提示并实现实时地理定位，适用于交互式场景。该模型采用全Transformer架构，利用Swin-Transformer对无人机与卫星图像进行联合特征编码，并使用无锚框检测头进行坐标回归。为更好捕捉跨模态与视图内依赖关系，我们在跨视图编码器中引入网格级稀疏专家混合（GMoE），根据每个网格的内容、尺度和来源自适应激活专用专家。同时采用热图监督的无锚框检测头，直接预测参考图像中的物体位置，避免预定义锚框带来的尺度偏差与匹配复杂性。在无人机至卫星任务中，SMGeo在IoU=0.25和mIoU指标上达到领先性能（测试集分别为87.51%、62.50%、61.45%），显著优于代表性方法DetGeo（61.97%、57.66%、54.05%）。消融实验表明，共享编码、查询引导融合与网格级稀疏专家混合均带来互补增益。
原文摘要 · Abstract (English)
Cross-view object Geo-localization aims to precisely pinpoint the same object across large-scale satellite imagery based on drone images. Due to significant differences in viewpoint and scale, coupled with complex background interference, traditional multi-stage "retrieval-matching" pipelines are prone to cumulative errors. To address this, we present SMGeo, a promptable end-to-end transformer-based model for object Geo-localization. This model supports click prompting and can output object Geo-localization in real time when prompted to allow for interactive use. The model employs a fully transformer-based architecture, utilizing a Swin-Transformer for joint feature encoding of both drone and satellite imagery and an anchor-free transformer detection head for coordinate regression. In order to better capture both inter-modal and intra-view dependencies, we introduce a grid-level sparse Mixture-of-Experts (GMoE) into the cross-view encoder, allowing it to adaptively activate specialized experts according to the content, scale and source of each grid. We also employ an anchor-free detection head for coordinate regression, directly predicting object locations via heat-map supervision in the reference images. This approach avoids scale bias and matching complexity introduced by predefined anchor boxes. On the drone-to-satellite task, SMGeo achieves leading performance in accuracy at IoU=0.25 and mIoU metrics (e.g., 87.51%, 62.50%, and 61.45% in the test set, respectively), significantly outperforming representative methods such as DetGeo (61.97%, 57.66%, and 54.05%, respectively). Ablation studies demonstrate complementary gains from shared encoding, query-guided fusion, and grid-level sparse mixture-of-experts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。