用旋转相机编码生成任意长度视频重拍，让镜头动得更准、背景更稳。
ReDirector: Creating Any-Length Video Retakes with Rotary Camera Encoding
- 将相机运动信息融入旋转位置编码，实现动态视频重拍控制
- 在不同镜头轨迹和长度下，对象定位更准、背景更清晰
- 适合需要精准镜头控制的视频编辑与虚拟拍摄场景
我们提出 ReDirector，一种基于相机控制的动态可变长视频重拍生成方法。针对以往工作中旋转位置编码（RoPE）的误用，我们通过对齐输入视频与目标重拍的时空位置进行修正。此外，引入旋转相机编码（RoCE），一种受相机条件调控的RoPE相位偏移机制，以捕捉并整合输入视频与目标重拍之间及内部的多视角关系。通过将相机条件嵌入RoPE，该方法可泛化至分布外的相机轨迹和视频长度，在动态物体定位和静态背景保留方面表现更优。大量实验表明，该方法在多种镜头轨迹和长度下，显著提升了镜头可控性、几何一致性与视频质量。
原文摘要 · Abstract (English)
We present ReDirector, a novel camera-controlled video retake generation method for dynamically captured variable-length videos. In particular, we rectify a common misuse of RoPE in previous works by aligning the spatiotemporal positions of the input video and the target retake. Moreover, we introduce Rotary Camera Encoding (RoCE), a camera-conditioned RoPE phase shift that captures and integrates multi-view relationships within and across the input and target videos. By integrating camera conditions into RoPE, our method generalizes to out-of-distribution camera trajectories and video lengths, yielding improved dynamic object localization and static background preservation. Extensive experiments further demonstrate significant improvements in camera controllability, geometric consistency, and video quality across various trajectories and lengths.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。