构建首个遥感大模型数据集与评估基准，解决模型幻觉问题。
DDFAV: Remote Sensing Large Vision Language Models Dataset and Evaluation Benchmark
- 用数据增强和混合策略构建高质量遥感图文数据集
- 提出评估方法RSPOPE，验证多模型零样本能力
- 适合遥感、多模态模型研究者使用
随着大视觉语言模型（LVLMs）的快速发展，其在多模态任务中表现优异。然而，由于LVLMs易产生幻觉，且目前缺乏专门针对遥感领域的数据集和评估方法，导致其在遥感任务上性能不佳。为此，本文提出一个高质量的遥感LVLM数据集DDFAV，采用数据增强与数据混合策略构建。基于该数据集，筛选优质遥感图像生成训练指令集。最后，设计基于该数据集的遥感幻觉评估方法RSPOPE，评估不同LVLM的零样本能力。相关数据集、指令集及评估方法已开源，地址为https://github.com/HaodongLi2024/rspope。
原文摘要 · Abstract (English)
With the rapid development of large vision language models (LVLMs), these models have shown excellent results in various multimodal tasks. Since LVLMs are prone to hallucinations and there are currently few datasets and evaluation methods specifically designed for remote sensing, their performance is typically poor when applied to remote sensing tasks. To address these issues, this paper introduces a high quality remote sensing LVLMs dataset, DDFAV, created using data augmentation and data mixing strategies. Next, a training instruction set is produced based on some high-quality remote sensing images selected from the proposed dataset. Finally, we develop a remote sensing LVLMs hallucination evaluation method RSPOPE based on the proposed dataset and evaluate the zero-shot capabilities of different LVLMs. Our proposed dataset, instruction set, and evaluation method files are available at https://github.com/HaodongLi2024/rspope.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。