用图谱方法量化多模态大模型的幻觉，让不可靠输出变可测量。
Grounding the Ungrounded: A Spectral-Graph Framework for Quantifying Hallucinations in Multimodal LLMs
- 基于扩散动力学和图拉普拉斯谱分解，构建幻觉量化框架
- 提出温度相关的幻觉指标，能追踪提示词与时间的变化趋势
- 提供可解释的跨模态度量，适合评估与改进模型可靠性
多模态大模型中的幻觉问题严重威胁其可信度。本文提出一种基于扩散动力学的信息几何框架，通过多模态图拉普拉斯的谱分解嵌入模型输出，并以与真实语义流形的差距定义语义失真度量。我们推导了温度依赖的幻觉轮廓的Courant-Fischer边界，利用再生核希尔伯特空间（RKHS）特征模态，获得可解释的、模态感知的度量，可追踪提示与时间演进过程。该框架将幻觉重新定义为可量化且有界的问题，为评估与缓解提供了原则性基础。
原文摘要 · Abstract (English)
Hallucinations in LLMs--especially in multimodal settings--undermine reliability. We present a rigorous information-geometric framework, grounded in diffusion dynamics, to quantify hallucinations in MLLMs where model outputs are embedded via spectral decompositions of multimodal graph Laplacians, and their gaps to a truth manifold define a semantic distortion metric. We derive Courant-Fischer bounds on a temperature-dependent hallucination profile and use RKHS eigenmodes to obtain modality-aware, interpretable measures that track evolution over prompts and time. This reframes hallucination as quantifiable and bounded, providing a principled basis for evaluation and mitigation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。