通过添加云雾图案欺骗遥感视觉语言检索系统,让其错误引用天气信息。
From Clouds to Hallucinations: Atmospheric Retrieval Hijacking in Remote Sensing Vision-Language RAG

- 在遥感图像上叠加可学习的云雾模式,优化以误导检索结果。
- 使目标气象证据排名提升至前5位,准确率从0.71%升至43.29%。
- 攻击自然外观,适合研究遥感安全与多模态RAG漏洞者阅读。
多模态RAG系统依赖视觉-语言检索器将视觉查询与外部文本证据对齐。现有对抗研究主要针对检索库或记忆,而对遥感模型的攻击多聚焦于最终任务预测。遥感多模态RAG中证据检索阶段的输入空间威胁尚未被充分探索。为此,我们提出CloudWeb,一种仅修改输入图像的气象检索劫持攻击,保持检索器、生成器和知识库不变。CloudWeb在遥感图像上叠加参数化的云/霾模式,并以检索为导向的目标函数优化:将对抗图像嵌入拉向目标气象证据,抑制源场景证据,强制排名分离,并正则化自然性和覆盖率。据我们所知,这是首个针对遥感多模态RAG检索阶段的气象证据劫持研究。我们在七个数据集的遥感RAG基准上评估CloudWeb,涵盖五种CLIP风格检索器(如GeoRSCLIP、RemoteCLIP、OpenAI CLIP、OpenCLIP),以及下游视觉-语言生成器。在所有检索器上,CloudWeb均显著优于干净检索、手工设计的气象基线、随机云扰动及固定变体,成功将天气相关证据注入前5名结果。在GeoRSCLIP ViT-B/32上,Weather@5从0.71%提升至43.29%。下游生成进一步表现出明显的天气幻觉与语义偏移,表明检索阶段劫持可传播至最终RAG响应。这些发现揭示了一种现实故障模式:看似自然的气象变化可在生成前破坏证据检索。
原文摘要 · Abstract (English)
Multimodal RAG systems increasingly rely on vision-language retrievers to ground visual queries in external textual evidence. Existing adversarial studies on RAG mainly manipulate the retrieval corpus or memory, while attacks on vision-language and remote sensing models typically target end-task predictions. Input-space threats to the evidence retrieval stage of remote sensing multimodal RAG remain underexplored. To address this gap, we introduce CloudWeb, an atmospheric retrieval hijacking attack that modifies only the input image while keeping the retriever, generator, and knowledge base fixed at deployment. CloudWeb overlays parameterized cloud- and haze-like patterns on remote sensing images and optimizes them with a retrieval-oriented objective that pulls adversarial image embeddings toward target atmospheric evidence, suppresses source-scene evidence, enforces rank separation, and regularizes naturalness and coverage. To the best of our knowledge, this is the first study of retrieval-stage atmospheric evidence hijacking in remote sensing multimodal RAG. We evaluate CloudWeb on a seven-dataset remote sensing RAG benchmark with five CLIP-style retrievers, including GeoRSCLIP, RemoteCLIP, OpenAI CLIP, and OpenCLIP, together with downstream vision-language generators. Across retrievers, CloudWeb consistently outperforms clean retrieval, handcrafted atmospheric baselines, random cloud perturbations, and fixed variants in injecting weather-related evidence into top-ranked results. On GeoRSCLIP ViT-B/32, Weather@5 increases from 0.71\% to 43.29\%. Downstream generation further shows measurable weather hallucination and semantic shift, indicating that retrieval-stage hijacking can propagate to the final RAG response. These findings reveal a practical failure mode: natural-looking atmospheric changes can compromise evidence retrieval before generation begins.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。