用多轮偏好优化提升视频描述准确性和细节度。
Enhancing Multimodal LLM for Detailed and Accurate Video Captioning using Multi-Round Preference Optimization
- 采用多轮偏好优化，定期更新参考模型并重置低秩适配模块。
- 误差率降低40%（全局）和20%（局部），重复率下降35%。
- 适合关注视频理解与生成的开发者和研究者。
视频蕴含丰富信息，生成详尽准确的自然语言描述是视频理解的关键。本文提出 video-SALMONN 2，一种基于低秩适配（LoRA）的音视频大语言模型，通过定向偏好优化（DPO）提升配对音频视频的描述能力。我们设计新评估指标以衡量描述完整性和准确性，并引入多轮DPO（mrDPO）：每1,000步更新参考模型，合并并重初始化LoRA模块作为参数更新代理，并结合真实字幕引导以稳定训练过程。为缓解mrDPO导致的非描述能力灾难性遗忘，提出重生调优（rebirth tuning），使用mrDPO训练模型生成的字幕作为监督标签，微调预训练的LLM。实验表明，mrDPO显著提升video-SALMONN 2的描述准确率，全局误差率降低40%，局部误差率降低20%，重复率下降35%。最终模型仅70亿参数，超越GPT-4o与Gemini-1.5-Pro在视频描述任务上的表现，同时在同类模型中保持顶尖的视频问答性能。代码、模型检查点及数据将在录用后公开，演示地址见：https://video-salmonn-2.github.io。
原文摘要 · Abstract (English)
Videos contain a wealth of information, and generating detailed and accurate descriptions in natural language is a key aspect of video understanding. In this paper, we present video-SALMONN 2, an advanced audio-visual large language model (LLM) with low-rank adaptation (LoRA) designed for enhanced video (with paired audio) captioning through directed preference optimization (DPO). We propose new metrics to evaluate the completeness and accuracy of video descriptions, which are optimized using DPO. To further improve training, we introduce a novel multi-round DPO (mrDPO) approach, which involves periodically updating the DPO reference model, merging and re-initializing the LoRA module as a proxy for parameter updates after each training round (1,000 steps), and incorporating guidance from ground-truth video captions to stabilize the process. To address potential catastrophic forgetting of non-captioning abilities due to mrDPO, we propose rebirth tuning, which finetunes the pre-DPO LLM by using the captions generated by the mrDPO-trained model as supervised labels. Experiments show that mrDPO significantly enhances video-SALMONN 2's captioning accuracy, reducing global and local error rates by 40\% and 20\%, respectively, while decreasing the repetition rate by 35\%. The final video-SALMONN 2 model, with just 7 billion parameters, surpasses leading models such as GPT-4o and Gemini-1.5-Pro in video captioning tasks, while maintaining competitive performance to the state-of-the-art on widely used video question-answering benchmark among models of similar size. Upon acceptance, we will release the code, model checkpoints, and training and test data. Demos are available at \href{https://video-salmonn-2.github.io}{https://video-salmonn-2.github.io}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。