让视频镜头切换像电影导演一样有设计感,支持精准控制镜头运动和剪辑节奏。
ShotDirector: Directorially Controllable Multi-Shot Video Generation with Cinematographic Transitions
- 用6自由度相机参数+分层提示词,精确控制镜头运动与场景内容。
- 在4万条电影级剪辑数据上训练,实现自然流畅的镜头衔接。
- 适合影视创作、动画制作及可控视频生成研究者使用。
镜头切换在多镜头视频生成中至关重要,它决定了叙事表达与导演风格。然而,现有方法多关注镜头间的视觉一致性,忽视了剪辑设计与电影语言对连贯叙事的作用,常导致机械的镜头堆叠。为此,我们提出ShotDirector框架,融合参数级相机控制与分层编辑模式感知提示。具体而言,采用包含6-DoF位姿与内参设置的相机控制模块,实现精准相机信息注入;引入镜头感知掩码机制,结合专业剪辑模式生成分层提示,实现对镜头内容的细粒度控制。该设计将参数条件与高层语义引导有效结合,实现类电影的可控镜头转换。为支持训练与评估,我们构建了包含4万条样本的ShotWeaver40K数据集,涵盖电影级剪辑先验,并开发了针对可控多镜头视频生成的评价指标。大量实验验证了框架的有效性。
原文摘要 · Abstract (English)
Shot transitions play a pivotal role in multi-shot video generation, as they determine the overall narrative expression and the directorial design of visual storytelling. However, recent progress has primarily focused on low-level visual consistency across shots, neglecting how transitions are designed and how cinematographic language contributes to coherent narrative expression. This often leads to mere sequential shot changes without intentional film-editing patterns. To address this limitation, we propose ShotDirector, an efficient framework that integrates parameter-level camera control and hierarchical editing-pattern-aware prompting. Specifically, we adopt a camera control module that incorporates 6-DoF poses and intrinsic settings to enable precise camera information injection. In addition, a shot-aware mask mechanism is employed to introduce hierarchical prompts aware of professional editing patterns, allowing fine-grained control over shot content. Through this design, our framework effectively combines parameter-level conditions with high-level semantic guidance, achieving film-like controllable shot transitions. To facilitate training and evaluation, we construct ShotWeaver40K, a dataset that captures the priors of film-like editing patterns, and develop a set of evaluation metrics for controllable multi-shot video generation. Extensive experiments demonstrate the effectiveness of our framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。