提出新评估框架,同时检测视觉语言模型的物体与关系幻觉。
Unified Triplet-Level Hallucination Evaluation for Large Vision-Language Models
- 通过三元组结构评估模型生成内容的真假
- 发现关系幻觉比物体幻觉更严重
- 无需训练即可有效减少幻觉,适合模型开发者使用
尽管大型视觉语言模型在视觉-语言推理中表现优异,但可能生成图像中并不存在的幻觉内容。现有幻觉评估基准大多仅关注物体相关幻觉,而两个物体间的关系幻觉仍缺乏研究。为此,我们设计了一个统一框架,可同时衡量模型在物体和关系上的幻觉。该框架的核心思想是通过从模型输出中提取(物体, 关系, 物体)三元组进行评估,具有任务泛化性。基于此,我们构建了新的三元组级幻觉评估基准 Tri-HE,可同步研究物体与关系幻觉。在 Tri-HE 上的全面评估显示,现有大型视觉语言模型中关系幻觉问题比物体幻觉更为严重,揭示了一个此前被忽视的可靠性问题。此外,基于该发现,我们提出一种无需训练的简单方法,能有效缓解模型幻觉。数据集与代码已公开于 https://github.com/wujunjie1998/Tri-HE。
原文摘要 · Abstract (English)
Despite the outstanding performance in vision-language reasoning, Large Vision-Language Models (LVLMs) might generate hallucinated contents that do not exist in the given image. Most existing LVLM hallucination benchmarks are constrained to evaluate the object-related hallucinations. However, the potential hallucination on the relations between two objects, i.e., relation hallucination, still lacks investigation. To remedy that, we design a unified framework to measure the object and relation hallucination in LVLMs simultaneously. The core idea of our framework is to evaluate hallucinations via (object, relation, object) triplets extracted from LVLMs' responses, making it easily generalizable to different vision-language tasks. Based on our framework, we further introduce Tri-HE, a novel Triplet-level Hallucination Evaluation benchmark which can be used to study both object and relation hallucination at the same time. With comprehensive evaluations on Tri-HE, we observe that the relation hallucination issue is even more serious than object hallucination among existing LVLMs, highlighting a previously neglected problem towards reliable LVLMs. Moreover, based on our findings, we design a simple training-free approach that effectively mitigates hallucinations for LVLMs. Our dataset and code for the reproduction of our experiments are available publicly at https://github.com/wujunjie1998/Tri-HE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。