无需人工标注,模型自动生成优劣对比数据来提升视频理解能力
VideoSAVi: Self-Aligned Video Language Models without Human Supervision
- 用自评机制识别回答错误并生成更优答案,形成偏好对
- 在多个基准上表现提升,最高达6.8个百分点
- 不依赖外部模型或标注,计算成本低
近期视频大语言模型(Video-LLMs)在视频理解方面取得显著进展。当前偏好优化方法通常依赖专有API或人工标注的字幕生成偏好数据(即人类判断质量高低的输出对),用于训练模型实现视频与语言对齐,但这一过程成本高且耗时。为解决此问题,我们提出VideoSAVi(自对齐视频语言模型),一种无需外部监督的自训练流程,使视频大语言模型能直接从视频内容中学习。该方法包含自评机制,可识别模型初始回答中的推理错误,并生成改进版本,从而直接从视频内容构建偏好对。随后,VideoSAVi采用直接偏好优化(DPO)迭代训练模型,增强其时空推理能力。实验表明,VideoSAVi在多个基准上均取得显著提升:相较基线模型,在MVBench上提升4.2个百分点,在PerceptionTest上提升3.9,在挑战性EgoSchema数据集上提升6.8。本方法具有模型无关性,仅需32帧视频,为无外部依赖的自对齐视频理解提供了高效可行方向。
原文摘要 · Abstract (English)
Recent advances in video-large language models (Video-LLMs) have led to significant progress in video understanding. Current preference optimization methods often rely on proprietary APIs or human-annotated captions to generate preference data (i.e., pairs of model outputs ranked by quality or alignment with human judgment), which is then used to train models for video-language alignment. This approach is both costly and labor-intensive. To address this limitation, we introduce VideoSAVi (Self-Aligned Video Language Model), a self-training pipeline that enables Video-LLMs to learn from video content without external supervision. Our approach includes a self-critiquing mechanism that identifies reasoning errors in the model's initial responses and generates improved alternatives, creating preference pairs directly from video content. VideoSAVi then applies Direct Preference Optimization (DPO) to iteratively train the model using the preference data, thus enhancing its temporal and spatial reasoning for video understanding. Experiments show that VideoSAVi delivers significant improvements across multiple benchmarks, including a +4.2 percentage point gain on MVBench, +3.9 on PerceptionTest, and +6.8 on the challenging EgoSchema dataset compared to baseline models. Our model-agnostic approach is computationally efficient, requiring only 32 frames, offering a promising direction for self-aligned video understanding without reliance on external models or annotations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。