arXiv:2607.06032cs.IR2026-07

让遥感图文检索知道自己的不确定,提高复杂场景下的可靠性。

Uncertainty-Aware Cross-Modal Remote Sensing Image-Text Retrieval via Evidential Learning

  • 用证据学习建模图文对应关系,量化每条查询的不确定性。
  • 在多种遥感退化条件下,检索准确率优于现有方法。
  • 适合处理传感器/大气干扰和专业术语差异的复杂遥感场景。

在跨模态遥感图像-文本检索(CMRSITR)中,测试时的遥感图像与文本描述可能因传感器和大气引起的图像退化以及文本侧遥感词汇异质性而偏离理想条件。现有方法在测试时对每个查询都假设完全确定,无法区分不同查询间的不确定性,导致结果不可靠。为此,本文提出基于证据学习的CMRSITR方法(ELC),在训练阶段,通过证据学习(EDL)将遥感图像与文本描述的跨模态对应关系建模为狄利克雷分布,从而获得每条查询的不确定性。进一步引入不确定性-正确性对齐学习(UCL),使错误检索对应高不确定性,正确检索对应低不确定性。同时,通过内模态关系学习(RL),从预训练教师编码器中提取内模态相似性结构,提升可训练编码器建模的判别能力。测试阶段,将估计的不确定性与固定拒识比例对应的阈值比较:低不确定性查询直接返回,高不确定性查询则通过遥感感知的测试时增强(RS-TTA)进行精炼。实验表明,ELC在多项遥感特定退化条件下(包括传感器与大气相关图像扰动、遥感词汇异质性)均达到先进水平,并展现出更强的鲁棒性。

原文摘要 · Abstract (English)

In cross-modal remote sensing image-text retrieval (CMRSITR), test-time remote sensing (RS) images and textual descriptions may deviate from well-curated benchmark conditions due to sensor- and atmosphere-related image degradations and text-side RS-vocabulary heterogeneity. Under such non-ideal conditions, existing CMRSITR methods may produce unreliable retrieval results because they perform retrieval with full certainty for each query and do not distinguish the varying uncertainty across queries. To address this issue, we propose an evidential learning-based CMRSITR (ELC) method for uncertainty-aware retrieval. During the training phase of ELC, evidential learning (EDL) is employed to model the inter-modal correspondences between RS images and textual descriptions as Dirichlet distributions, from which the uncertainty of each query can be obtained. Based on the EDL outputs, uncertainty-correctness alignment learning (UCL) is introduced to align the estimated uncertainty with retrieval correctness, encouraging high uncertainty for incorrect retrieval and low uncertainty for correct retrieval. Furthermore, intra-modal relationship learning (RL) distills the intra-modal similarity structure from pretrained mentor encoders for the trainable encoders, thereby making the Dirichlet distributions modeled by EDL more discriminative. In the test phase of ELC, the estimated uncertainty is compared with a threshold determined by a fixed deferral ratio, where low-uncertainty queries are directly returned and high-uncertainty queries are refined by RS-aware test-time augmentation (RS-TTA). Experimental results demonstrate that ELC achieves competitive retrieval performance compared with state-of-the-art CMRSITR methods and provides stronger robustness under the evaluated RS-specific degradations, including sensor- and atmosphere-related image perturbations and RS-vocabulary heterogeneity.

遥感检索不确定性证据学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。