arXiv:2606.15162cs.CV2026-06被引 3

GeoStream实现精准相机控制的流式视频生成,解决视角超出初始范围时的误差问题。

GeoStream: Toward Precise Camera Controlled Streaming Video Generation

论文配图:GeoStream: Toward Precise Camera Controlled Streaming Video Generation
图 1 · 摘自论文原文
  • 用自更新3D缓存实时提供几何条件,支持连续视角变化
  • 在多个基准上相机轨迹误差降低40%以上,视觉质量显著提升
  • 适合需要精确相机控制的交互式视频生成场景

准确的交互式相机控制对基于视频的世界模型至关重要,但现有方法大多隐式学习相机运动,在分布外轨迹下控制精度差。显式几何条件虽能提升可控性,但现有方法为非自回归,依赖初始帧构建的静态3D缓存,一旦视角超出原始视锥即失效。我们提出GeoStream,一种支持自回归流式视频生成的精确度量级相机控制框架。该方法维护一个由模型自身输出在线刷新的3D缓存:从最新生成帧估计深度,反投影至3D空间,并重新投影至目标视图,生成点重投影作为后续合成的几何条件。训练阶段也通过学生自身生成帧渲染条件,实现完全在线策略蒸馏,自然对齐训练与推理的条件分布。相比使用离策略噪声的方法,本方案使模型在推理时面对的是真实误差分布,有效缓解了自回归漂移及因缓存来自生成结果而引发的二阶几何反馈环。定量与定性结果显示,该方法显著提升了相机可控性。

原文摘要 · Abstract (English)

Accurate interactive camera control is essential for video-based world models, but most existing approaches learn camera motion implicitly, leading to inaccurate control under out-of-distribution trajectories. Explicit geometric conditioning improves controllability, but existing methods are non-autoregressive and rely on a static 3D cache built from an initial frame, which becomes ineffective once the viewpoint moves beyond the original frustum. We propose GeoStream, a framework that enables precise metric-scale camera control in autoregressive streaming video generation. Our method maintains a self-refreshing 3D cache that is periodically updated online from the model's own outputs: we estimate depth from the most recently generated frame, unproject to 3D, and reproject into the target view to produce point reprojections as geometric conditioning for subsequent synthesis. By the same principle, the conditioning seen during training is also rendered from the student's own generated frames, yielding a fully on-policy distillation that naturally aligns the train and inference conditioning distributions. Unlike prior work that uses off-policy condition noising, our approach trains the model against the exact error distribution it encounters at inference, mitigating both standard autoregressive drift and the second-order geometric feedback loop that arises when the cache itself is derived from generated outputs. Quantitative and qualitative results show that our approach substantially improves camera controllability.

视频生成相机控制自回归

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。