arXiv:2507.16869cs.GRcs.CV2025-07综述被引 87

综述可控视频生成方法,解决文本指令难以精确控制视频的问题。

Controllable Video Generation: A Survey

论文配图:Controllable Video Generation: A Survey
图 1 · 摘自论文原文
  • 引入相机运动、姿态、深度图等非文本条件增强生成控制力。
  • 系统梳理扩散模型中多类型条件如何融入去噪过程实现可控生成。
  • 适合关注视频生成可控性与多模态输入融合的研究者参考。

随着AI生成内容(AIGC)的快速发展,视频生成已成为最具活力和影响力的研究方向之一。特别是视频生成基础模型的进步,推动了对更精准表达用户意图的可控视频生成方法的需求。现有大部分基础模型聚焦于文本到视频生成,但仅靠文本提示往往无法充分描述复杂、多模态且精细的用户需求,导致当前模型难以实现精确控制。为此,近期研究探索将相机运动、深度图、人体姿态等非文本条件整合进预训练视频生成模型,以扩展其能力并实现更强的可控性合成。这些方法旨在提升AIGC驱动视频生成系统的灵活性与实用性。本文系统综述了可控视频生成领域,涵盖理论基础与最新进展。首先介绍核心概念及常用开源视频生成模型;接着重点分析视频扩散模型中的控制机制,探讨不同条件如何嵌入去噪过程以引导生成;最后根据所利用的控制信号类型,将现有方法分为单条件生成、多条件生成与通用可控生成三类。完整文献列表请访问我们的精选资源库:https://github.com/mayuelala/Awesome-Controllable-Video-Generation。

原文摘要 · Abstract (English)

With the rapid development of AI-generated content (AIGC), video generation has emerged as one of its most dynamic and impactful subfields. In particular, the advancement of video generation foundation models has led to growing demand for controllable video generation methods that can more accurately reflect user intent. Most existing foundation models are designed for text-to-video generation, where text prompts alone are often insufficient to express complex, multi-modal, and fine-grained user requirements. This limitation makes it challenging for users to generate videos with precise control using current models. To address this issue, recent research has explored the integration of additional non-textual conditions, such as camera motion, depth maps, and human pose, to extend pretrained video generation models and enable more controllable video synthesis. These approaches aim to enhance the flexibility and practical applicability of AIGC-driven video generation systems. In this survey, we provide a systematic review of controllable video generation, covering both theoretical foundations and recent advances in the field. We begin by introducing the key concepts and commonly used open-source video generation models. We then focus on control mechanisms in video diffusion models, analyzing how different types of conditions can be incorporated into the denoising process to guide generation. Finally, we categorize existing methods based on the types of control signals they leverage, including single-condition generation, multi-condition generation, and universal controllable generation. For a complete list of the literature on controllable video generation reviewed, please visit our curated repository at https://github.com/mayuelala/Awesome-Controllable-Video-Generation.

视频生成可控生成扩散模型多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。