用稀疏提示生成连贯视频，通过密集运动场和注意力增强实现稳定时序一致性。
STANCE: Motion Coherent Video Generation Via Sparse-to-Dense Anchored Encoding
- 将稀疏2D提示转为稠密2.5D运动场，提升引导精度。
- 在首个帧锚定少量运动令牌，结合旋转位置编码保持语义显著性。
- 无需逐帧轨迹标注，适合需高时序一致性的视频生成任务。
视频生成虽在视觉表现上取得显著进展，但保持物体运动与交互的一致性仍具挑战。我们识别出两个实际瓶颈：(i) 人类提供的运动提示（如小范围2D图）在编码后常坍缩为过少的有效令牌，削弱引导作用；(ii) 在单一头中同时优化外观与运动，容易偏向纹理而忽视时序一致性。本文提出STANCE，一种图像到视频的框架，通过两个简单组件解决上述问题。首先引入实例提示（Instance Cues），即像素对齐的控制信号，通过平均每个实例的光流并结合实例掩码上的单目深度，将稀疏、可编辑的提示转化为稠密2.5D（相机相对）运动场，相比2D箭头输入降低深度歧义，且仍易于使用。其次，采用密集旋转变换位置编码（Dense RoPE），为少数锚定在首帧的运动令牌赋予空间可定位的旋转嵌入，保留其在令牌空间中的显著性。结合联合RGB + 辅助图预测（分割或深度），模型在保持结构的同时由RGB处理外观，稳定优化过程并提升时序一致性，无需每帧轨迹脚本。
原文摘要 · Abstract (English)
Video generation has recently made striking visual progress, but maintaining coherent object motion and interactions remains difficult. We trace two practical bottlenecks: (i) human-provided motion hints (e.g., small 2D maps) often collapse to too few effective tokens after encoding, weakening guidance; and (ii) optimizing for appearance and motion in a single head can favor texture over temporal consistency. We present STANCE, an image-to-video framework that addresses both issues with two simple components. First, we introduce Instance Cues -- a pixel-aligned control signal that turns sparse, user-editable hints into a dense 2.5D (camera-relative) motion field by averaging per-instance flow and augmenting with monocular depth over the instance mask. This reduces depth ambiguity compared to 2D arrow inputs while remaining easy to use. Second, we preserve the salience of these cues in token space with Dense RoPE, which tags a small set of motion tokens (anchored on the first frame) with spatial-addressable rotary embeddings. Paired with joint RGB \(+\) auxiliary-map prediction (segmentation or depth), our model anchors structure while RGB handles appearance, stabilizing optimization and improving temporal coherence without requiring per-frame trajectory scripts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。