用镜头变化生成视频,零训练即可跟拍新视角。
Warp-as-History: Generalizable Camera-Controlled Video Generation from One Training Video

- 将镜头扭曲转为伪历史序列,对齐目标帧位置编码
- 零训练下实现镜头轨迹跟随,仅需单个标注视频微调
- 无需测试时优化,适合快速部署到新视频场景
相机控制视频生成已取得显著进展,可使生成视频沿预定视角轨迹运动。但现有方法通常依赖相机编码器、控制分支或注意力与位置编码修改来学习相机特定条件,常需在大规模带相机标注的视频上进行后训练。训练免费替代方案虽避免后训练,但往往将成本转移至测试时优化或额外去噪引导。本文提出Warp-as-History:一个简单接口,将相机引起的形变转化为相机扭曲的伪历史序列,通过目标帧位置对齐和可见标记选择实现重建。给定目标相机轨迹,从过往观测构建相机扭曲伪历史,并输入模型的视觉历史路径。关键在于将其位置编码与正在去噪的目标帧对齐,并移除无有效源观测的扭曲历史标记。无需任何训练、架构修改或测试时优化,该接口揭示了冻结视频生成模型具备非平凡的零样本镜头轨迹跟随能力。此外,在仅一个带相机标注视频上进行轻量级离线LoRA微调,进一步提升性能并泛化至未见视频,在不使用测试时优化或目标视频适配的情况下,提升镜头跟随精度、视觉质量和运动动态。在多种数据集上的大量实验验证了方法的有效性。
原文摘要 · Abstract (English)
Camera-controlled video generation has made substantial progress, enabling generated videos to follow prescribed viewpoint trajectories. However, existing methods usually learn camera-specific conditioning through camera encoders, control branches, or attention and positional-encoding modifications, which often require post-training on large-scale camera-annotated videos. Training-free alternatives avoid such post-training, but often shift the cost to test-time optimization or extra denoising-time guidance. We propose Warp-as-History, a simple interface that turns camera-induced warps into camera-warped pseudo-history with target-frame positional alignment and visible-token selection. Given a target camera trajectory, we construct camera-warped pseudo-history from past observations and feed it through the model's visual-history pathway. Crucially, we align its positional encoding with the target frames being denoised and remove warped-history tokens without valid source observations. Without any training, architectural modification, or test-time optimization, this interface reveals a non-trivial zero-shot capability of a frozen video generation model to follow camera trajectories. Moreover, lightweight offline LoRA finetuning on only one camera-annotated video further improves this capability and generalizes to unseen videos, improving camera adherence, visual quality, and motion dynamics without test-time optimization or target-video adaptation. Extensive experiments on diverse datasets confirm the effectiveness of our method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。