不训练即可减少遥感图文问答中的幻觉，提升模型准确性
Seeing Clearly without Training: Mitigating Hallucinations in Multimodal LLMs for Remote Sensing
- 利用模型自身注意力机制，在推理时逐步定位细粒度目标
- 在多个遥感问答任务中，显著降低事实与逻辑幻觉率
- 无需训练，适用于各类现有多模态大模型，适合遥感应用开发者
多模态大语言模型在遥感视觉问答任务中存在严重幻觉问题，主要源于大场景下的视觉定位失败或对细微目标的误理解。为系统分析此类问题，我们提出 RSHBench，一个基于协议的细粒度幻觉诊断基准。为缓解由定位错误引发的事实性幻觉，我们进一步提出 RADAR——一种无需训练的推理方法，通过挖掘 MLLMs 内在注意力，实现测试时的渐进式定位与细粒度局部推理。在多种 MLLM 上的大量实验表明，RADAR 能持续提升遥感问答性能，并有效减少事实与逻辑幻觉。代码与数据将公开于：https://github.com/MiliLab/RADAR
原文摘要 · Abstract (English)
Multimodal large language models (MLLMs) suffer from pronounced hallucinations in remote sensing visual question-answering (RS-VQA), primarily caused by visual grounding failures in large-scale scenes or misinterpretation of fine-grained small targets. To systematically analyze these issues, we introduce RSHBench, a protocol-based benchmark for fine-grained diagnosis of factual and logical hallucinations. To mitigate grounding-induced factual hallucinations, we further propose Relative Attention-Driven Actively Reasoning (RADAR), a training-free inference method that leverages intrinsic attention in MLLMs to guide progressive localization and fine-grained local reasoning at test time. Extensive experiments across diverse MLLMs demonstrate that RADAR consistently improves RS-VQA performance and reduces both factual and logical hallucinations. Code and data will be publicly available at: https://github.com/MiliLab/RADAR
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。