arXiv:2604.00455cs.CVcs.AI2026-04被引 1

通过强化首个词的逻辑值,抑制大模型视觉幻觉。

First Logit Boosting: Visual Grounding Method to Mitigate Object Hallucination in Large Vision-Language Models

  • 存储首个生成词的逻辑值并叠加到后续预测中
  • 在多个任务和模型上显著降低幻觉率
  • 无需训练、开销极小,适合实时系统

近期大型视觉语言模型(LVLM)在需同时理解视觉与语言输入的多模态任务中表现优异,但对象幻觉——即在回答中生成不存在的对象——仍是持续存在的挑战。尽管已有重训练或外部定位等方法,仍面临数据成本高或结构复杂的问题。无训练方法如对比解码(CD)虽更经济,却存在长期衰减问题:生成过程中视觉信息减弱,语言先验主导。本文提出首词逻辑增强(FLB),一种简单有效的无训练技术,旨在缓解该衰减。FLB保存首个生成词的逻辑值,并将其加至后续词预测中,有效维持整个生成过程中的视觉信息。实验表明,FLB(1)使首词所携带的视觉信息贯穿生成全程;(2)通过稳定“the”词的出现,抑制幻觉词。在多种任务、基准及骨干模型上,FLB均显著降低对象幻觉,且推理开销可忽略不计,适用于实时多模态系统。代码已开源。

原文摘要 · Abstract (English)

Recent Large Vision-Language Models (LVLMs) have demonstrated remarkable performance across various multimodal tasks that require understanding both visual and linguistic inputs. However, object hallucination -- the generation of nonexistent objects in answers -- remains a persistent challenge. Although several approaches such as retraining and external grounding methods have been proposed to mitigate this issue, they still suffer from high data costs or structural complexity. Training-free methods such as Contrastive Decoding (CD) are more cost-effective, avoiding additional training or external models, but still suffer from long-term decay, where visual grounding weakens and language priors dominate as the generation progresses. In this paper, we propose First Logit Boosting (FLB), a simple yet effective training-free technique designed to alleviate long-term decay in LVLMs. FLB stores the logit of the first generated token and adds it to subsequent token predictions, effectively mitigating long-term decay of visual information. We observe that FLB (1) sustains the visual information embedded in the first token throughout generation, and (2) suppresses hallucinated words through the stabilizing effect of the ``The'' token. Experimental results show that FLB significantly reduces object hallucination across various tasks, benchmarks, and backbone models. Notably, it causes negligible inference overhead, making it highly applicable to real-time multimodal systems. Code is available at https://github.com/jiwooha20/FLB

视觉定位幻觉抑制无训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。