arXiv:2601.04068cs.CVcs.AI2026-01中稿 · CVPR被引 3

通过局部细节优化,让视频生成模型更懂人类偏好。

Mind the Generative Details: Direct Localized Detail Preference Optimization for Video Diffusion Models

  • 用真实视频做正样本,局部损坏生成负样本。
  • 在损坏区域训练,提升视频质量和连贯性。
  • 无需人工标注或额外评分模型,适合快速迭代。

将文本到视频的扩散模型对齐人类偏好对于生成高质量视频至关重要。现有直接偏好优化(DPO)方法依赖多样本排序和特定任务的评判模型,效率低下且常产生模糊的全局监督信号。为此,我们提出一种名为 LocalDPO 的新后训练框架,从真实视频中构建局部偏好对,并在时空区域层面进行对齐优化。我们设计了一个自动化流程,每个提示仅需一次推理即可生成偏好对,无需外部评判模型或人工标注。具体地,将高质量真实视频作为正样本,通过随机时空掩码局部破坏并仅用冻结的基础模型恢复掩码区域生成负样本。训练时引入区域感知的 DPO 损失,限制偏好学习仅在被破坏区域进行,实现快速收敛。在 Wan2.1 与 CogVideoX 数据集上的实验表明,LocalDPO 在视频保真度、时间连贯性和人类偏好评分方面均优于其他后训练方法,建立了一种更高效、更精细的视频生成对齐范式。代码已开源:https://github.com/1170300714/Local-DPO。

原文摘要 · Abstract (English)

Aligning text-to-video diffusion models with human preferences is crucial for generating high-quality videos. Existing Direct Preference Otimization (DPO) methods rely on multi-sample ranking and task-specific critic models, which is inefficient and often yields ambiguous global supervision. To address these limitations, we propose LocalDPO, a novel post-training framework that constructs localized preference pairs from real videos and optimizes alignment at the spatio-temporal region level. We design an automated pipeline to efficiently collect preference pair data that generates preference pairs with a single inference per prompt, eliminating the need for external critic models or manual annotation. Specifically, we treat high-quality real videos as positive samples and generate corresponding negatives by locally corrupting them with random spatio-temporal masks and restoring only the masked regions using the frozen base model. During training, we introduce a region-aware DPO loss that restricts preference learning to corrupted areas for rapid convergence. Experiments on Wan2.1 and CogVideoX demonstrate that LocalDPO consistently improves video fidelity, temporal coherence and human preference scores over other post-training approaches, establishing a more efficient and fine-grained paradigm for video generator alignment.The code is available at https://github.com/1170300714/Local-DPO.

视频生成扩散模型偏好优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。