提出SkyPart模型,提升无人机跨视角定位在恶劣天气下的准确性
Weather-Robust Cross-View Geo-Localization via Prototype-Based Semantic Part Discovery

- 通过原型竞争机制对图像块进行语义部件分组,分离布局与纹理
- 训练时用高度条件调制,使推理时嵌入不受高度影响,提升鲁棒性
- 新损失函数自动平衡多目标,适合无重排序的快速定位场景
跨视角地理定位(CVGL)将倾斜无人机视图与地理参考卫星影像匹配,是GNSS失效时自主导航的关键替代方案。尽管近期进展显著,仍存在三大局限:(1) 全局描述符将图像块网格压缩为单一向量,未能分离视图差异中的布局与纹理;(2) 高度相关的尺度变化保留在学习嵌入中,未被消除;(3) 多目标训练依赖人工调参的损失权重,且各损失梯度尺度不兼容。本文提出SkyPart,一种轻量级可替换的基于块的视觉变压器(ViT)头部,实现图像块网格上的显式部件分组。其包含四个理论驱动组件:(i) 通过单次余弦分配让可学习原型竞争图像块令牌;(ii) 仅在训练时应用高度条件线性调制,使推理时嵌入与高度无关;(iii) 对活跃原型使用图注意力读出;(iv) 采用肯德尔不确定性加权多目标损失,其平稳点为帕累托平稳点。在26.95M参数和22.14 GFLOPs下,SkyPart是当前顶尖方法中最小的,并在SUES-200、University-1652和DenseUAV数据集上以单次前向、无重排序、无测试时增强协议达到新最优性能。其相对于最强基线的优势在十种天气扰动(WeatherPrompt)条件下进一步扩大。
原文摘要 · Abstract (English)
Cross-view geo-localization (CVGL), which matches an oblique drone view to a geo-referenced satellite tile, has emerged as a key alternative for autonomous drone navigation when GNSS signals are jammed, spoofed, or unavailable. Despite strong recent progress, three limitations persist: (1) global-descriptor designs compress the patch grid into a single vector without separating layout from texture across the view gap; (2) altitude-related scale variation is retained in the learned embedding rather than marginalized; and (3) multi-objective training relies on hand-tuned scalars over losses on incompatible gradient scales. We propose SkyPart, a lightweight swappable head for patch-based vision transformers (ViTs) that institutes explicit part grouping over the patch grid. SkyPart has four theory-grounded components: (i) learnable prototypes competing for patch tokens via single-pass cosine assignment; (ii) altitude-conditioned linear modulation applied only during training, making the retrieval embedding altitude-free at inference; (iii) a graph-attention readout over active prototypes; and (iv) a Kendall uncertainty-weighted multi-objective loss whose stationary points are Pareto-stationary. At 26.95M parameters and 22.14 GFLOPs, SkyPart is the smallest among top-performing methods and sets a new state of the art on SUES-200, University-1652, and DenseUAV under a single-pass, no-re-ranking, no-TTA protocol. Its advantage over the strongest baseline widens under the ten-condition WeatherPrompt corruption benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。