提出无需训练的解码方法，让视频大模型回答更符合时间逻辑。
SEASON: Mitigating Temporal Hallucination in Video Large Language Models via Self-Diagnostic Contrastive Decoding
- 通过动态诊断每一步生成内容的幻觉倾向，自适应调整输出。
- 在三个幻觉检测基准上超越现有方法，提升视频理解通用性能。
- 适合关注视频时序一致性、追求零成本改进的开发者使用。
视频大语言模型在视频理解任务中表现突出，但仍难以有效感知和利用视频中的丰富时序信息，导致回复常出现时间不一致或因果不合理的问题，引发严重幻觉。与以往聚焦空间幻觉（如物体错配）的研究不同，时序推理仍相对未被充分探索。为此，本文提出无需训练的自诊断对比解码方法SEASON，通过动态诊断每个输出标记的幻觉倾向，并针对其对应的时序与空间负样本进行自适应对比解码，以增强输出的时间与空间忠实性。大量实验表明，SEASON在三个幻觉检测基准上优于所有现有无训练幻觉缓解方法，同时在四个通用视频理解基准上进一步提升了视频大模型的表现。代码将在论文接受后公开。
原文摘要 · Abstract (English)
Video Large Language Models (VideoLLMs) have shown remarkable progress in video understanding. However, these models still struggle to effectively perceive and exploit rich temporal information in videos when responding to user queries. Therefore, they often generate descriptions of events that are temporal inconsistent or causally implausible, causing severe hallucination issues. While most prior studies have focused on spatial hallucinations (e.g. object mismatches), temporal reasoning in video understanding remains relatively underexplored. To address this issue, we propose Self-Diagnostic Contrastive Decoding (SEASON), a training-free method that adaptively enhances temporal and spatial faithfulness for each output token. It achieves this by dynamically diagnosing each token's hallucination tendency and applying adaptive contrastive decoding against its corresponding temporal and spatial negatives. Extensive experiments demonstrate that SEASON outperforms all existing training-free hallucination mitigation approaches on three hallucination examination benchmarks, while further improves VideoLLMs across four general video understanding benchmarks. The code will be released upon acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。