通过物体遮蔽与插值构建排序列表，有效减少视觉语言模型幻觉。
LPOI: Listwise Preference Optimization for Vision Language Models
- 基于物体遮蔽和图像插值生成渐进式完整图像序列。
- 在三个基准上显著降低幻觉率，提升模型性能。
- 仅需成对偏好数据，无需额外标注，适合实际部署。
对齐大型视觉语言模型（VLM）与人类偏好是一项挑战，现有方法如强化学习人类反馈（RLHF）和直接偏好优化（DPO）常过度依赖文本信息或加剧幻觉。尽管增强负样本可部分缓解此问题，但此前无研究在VLM中采用列表级偏好优化，因其构建列表级图像样本的复杂性与成本较高。本文提出LPOI，首个面向对象的列表级偏好优化方法，旨在减少VLM中的幻觉。LPOI识别并遮蔽图像中的关键物体，将遮蔽区域在正负样本间进行插值，形成一系列逐步完整的图像序列。模型被训练为按物体可见度递增排序这些图像，从而在保持视觉保真度的同时有效抑制幻觉。LPOI无需额外标注，仅需标准成对偏好数据，通过自动遮蔽与插值构建排序列表。在MMHalBench、AMBER和Object HalBench上的全面实验表明，LPOI在减少幻觉和提升性能方面优于现有方法。代码已开源：https://github.com/fatemehpesaran310/lpoi。
原文摘要 · Abstract (English)
Aligning large VLMs with human preferences is a challenging task, as methods like RLHF and DPO often overfit to textual information or exacerbate hallucinations. Although augmenting negative image samples partially addresses these pitfalls, no prior work has employed listwise preference optimization for VLMs, due to the complexity and cost of constructing listwise image samples. In this work, we propose LPOI, the first object-aware listwise preference optimization developed for reducing hallucinations in VLMs. LPOI identifies and masks a critical object in the image, and then interpolates the masked region between the positive and negative images to form a sequence of incrementally more complete images. The model is trained to rank these images in ascending order of object visibility, effectively reducing hallucinations while retaining visual fidelity. LPOI requires no extra annotations beyond standard pairwise preference data, as it automatically constructs the ranked lists through object masking and interpolation. Comprehensive experiments on MMHalBench, AMBER, and Object HalBench confirm that LPOI outperforms existing preference optimization methods in reducing hallucinations and enhancing VLM performance. We make the code available at https://github.com/fatemehpesaran310/lpoi.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。