arXiv:2507.19586cs.CLcs.AI2025-07EMNLP被引 7

提出评估与缓解大模型地理幻觉的新方法,提升地理推理可靠性。

Mitigating Geospatial Knowledge Hallucination in Large Language Models: Benchmarking and Dynamic Factuality Aligning

  • 构建基于知识图谱的地理幻觉评测框架,实现精准量化
  • 20个先进大模型中发现普遍地理错误,动态对齐使性能提升29.6%
  • 适合地理信息、AI可信性研究者使用

大语言模型(LLMs)具备丰富的世界知识,包括地理空间知识,已成功应用于出行预测和社情指标预测等任务。然而,这些模型常生成不准确的地理信息,导致地理幻觉(即地理信息的错误或不一致表示),影响其可靠性。尽管通用知识幻觉已被广泛研究,但地理幻觉的系统评估与缓解仍缺乏探索。为此,我们提出一个全面的地理幻觉评估框架,利用结构化地理知识图谱进行受控评估。在20个先进大模型上进行广泛测试后,揭示了其地理知识中的幻觉现象。基于此,我们引入一种基于Kahneman-Tversky优化(KTO)的动态事实对齐方法,有效缓解地理幻觉,在新提出的基准上性能提升超29.6%。大量实验表明,该评测框架与学习算法显著提升了模型在地理知识与推理任务中的可信度。

原文摘要 · Abstract (English)

Large language models (LLMs) possess extensive world knowledge, including geospatial knowledge, which has been successfully applied to various geospatial tasks such as mobility prediction and social indicator prediction. However, LLMs often generate inaccurate geospatial knowledge, leading to geospatial hallucinations (incorrect or inconsistent representations of geospatial information) that compromise their reliability. While the phenomenon of general knowledge hallucination in LLMs has been widely studied, the systematic evaluation and mitigation of geospatial hallucinations remain largely unexplored. To address this gap, we propose a comprehensive evaluation framework for geospatial hallucinations, leveraging structured geospatial knowledge graphs for controlled assessment. Through extensive evaluation across 20 advanced LLMs, we uncover the hallucinations in their geospatial knowledge. Building on these insights, we introduce a dynamic factuality aligning method based on Kahneman-Tversky Optimization (KTO) to mitigate geospatial hallucinations in LLMs, leading to a performance improvement of over 29.6% on the proposed benchmark. Extensive experimental results demonstrate the effectiveness of our benchmark and learning algorithm in enhancing the trustworthiness of LLMs in geospatial knowledge and reasoning tasks.

地理知识幻觉缓解大模型可信推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。