提出新方法评估视觉语言模型中的对象幻觉，更全面捕捉错误生成。
Evaluating Hallucination in Large Vision-Language Models based on Context-Aware Object Similarities
- 结合物体统计与描述中语义关系，识别幻觉对象。
- 发现未在训练数据中的新对象也常被错误生成，现有方法会忽略。
- 分析生成顺序和语义原因，帮助理解幻觉成因，适合模型开发者参考。
尽管大型视觉语言模型（LVLMs）在多模态任务中表现优异，但仍普遍存在幻觉问题。其中一类重要幻觉是对象幻觉，即模型生成与输入图像不符的对象。现有方法通常通过检测生成描述中幻觉对象的比例来量化此类问题，或基于高频对象、共现对象等统计信息，以二元提问方式直接查询模型。本文提出上下文感知的物体相似性（CAOS），融合物体统计信息与生成描述中对象间的语义关系，实现对幻觉的更精准识别。传统方法仅关注训练数据中的预设对象集合，忽略了不在该集合内的生成对象，导致评估不足。为此，我们引入语言模型进行对象识别，检测潜在的域外幻觉对象，并使用多模型集成验证其在图像中的存在性。此外，CAOS还分析对象生成的时序动态，探究出现顺序对幻觉的影响，并利用词嵌入模型分析幻觉背后的语义原因。整体上，该框架系统化地识别与解释了LVLM的幻觉倾向。
原文摘要 · Abstract (English)
Despite their impressive performance on multi-modal tasks, large vision-language models (LVLMs) tend to suffer from hallucinations. An important type is object hallucination, where LVLMs generate objects that are inconsistent with the images shown to the model. Existing works typically attempt to quantify object hallucinations by detecting and measuring the fraction of hallucinated objects in generated captions. Additionally, more recent work also measures object hallucinations by directly querying the LVLM with binary questions about the presence of likely hallucinated objects based on object statistics like top-k frequent objects and top-k co-occurring objects. In this paper, we present Context-Aware Object Similarities (CAOS), a novel approach for evaluating object hallucination in LVLMs using object statistics as well as the generated captions. CAOS uniquely integrates object statistics with semantic relationships between objects in captions and ground-truth data. Moreover, existing approaches usually only detect and measure hallucinations belonging to a predetermined set of in-domain objects (typically the set of all ground-truth objects for the training dataset) and ignore generated objects that are not part of this set, leading to under-evaluation. To address this, we further employ language model--based object recognition to detect potentially out-of-domain hallucinated objects and use an ensemble of LVLMs for verifying the presence of such objects in the query image. CAOS also examines the sequential dynamics of object generation, shedding light on how the order of object appearance influences hallucinations, and employs word embedding models to analyze the semantic reasons behind hallucinations. CAOS aims to offer a nuanced understanding of the hallucination tendencies of LVLMs by providing a systematic framework to identify and interpret object hallucinations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。