提出新方法识别遥感图文检索中的虚假匹配对
PMPGuard: Catching Pseudo-Matched Pairs in Remote Sensing Image-Text Retrieval
- 用门控注意力与正负感知机制动态调节跨模态信息流
- 在三个遥感数据集上达到当前最优性能
- 适合处理真实场景中图文错配问题的研究者
遥感图像-文本检索在真实数据集中面临伪匹配对（PMPs）的挑战，即语义不匹配或弱对齐的图文对，干扰可靠跨模态对齐的学习。为此，我们提出一种新检索框架，利用跨模态门控注意力和正负感知注意力机制，缓解此类噪声关联的影响。门控模块动态调节跨模态信息流，而感知机制在对齐学习中显式区分有效（正）线索与误导（负）线索。在三个基准遥感数据集（RSICD、RSITMD、RS5M）上的大量实验表明，该方法持续取得领先性能，证明其在处理真实场景中的错配与伪匹配对方面的鲁棒性与有效性。
原文摘要 · Abstract (English)
Remote sensing (RS) image-text retrieval faces significant challenges in real-world datasets due to the presence of Pseudo-Matched Pairs (PMPs), semantically mismatched or weakly aligned image-text pairs, which hinder the learning of reliable cross-modal alignments. To address this issue, we propose a novel retrieval framework that leverages Cross-Modal Gated Attention and a Positive-Negative Awareness Attention mechanism to mitigate the impact of such noisy associations. The gated module dynamically regulates cross-modal information flow, while the awareness mechanism explicitly distinguishes informative (positive) cues from misleading (negative) ones during alignment learning. Extensive experiments on three benchmark RS datasets, i.e., RSICD, RSITMD, and RS5M, demonstrate that our method consistently achieves state-of-the-art performance, highlighting its robustness and effectiveness in handling real-world mismatches and PMPs in RS image-text retrieval tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。