让视频世界模型在运行中实时修改动作,不重播已生成内容
ActionSplice: In-Flight Action Editing for Interactive World Models

- 提出反事实状态迁移机制,动态修正中间状态
- 动作修改后减少75.9%的视觉误差,速度提升2.73倍
- 适合需要实时交互的视频生成与模拟系统
块自回归视频世界模型通常将每个生成块依赖于单一动作。采样过程中接收到的新动作必须等待下一区块,或基于前一动作产生的状态进行后续求解,或触发回滚并重复已完成计算。我们提出ActionSplice,将该问题建模为反事实状态迁移(CST)。轻量级校正器在相同求解步骤中,将中断的骨干网络表示向由新动作诱导的匹配状态转移。世界模型与采样器保持不变,无需重播已完成评估即可恢复采样。其中,重定向变体CST*R更新整个活跃块,时间拼接变体CST*T保留时间前缀,仅更新后缀。在minWM-Wan Action2V和HY-WM1.5数据集上,CST*R相比直接条件替换,分别降低61.5%和75.9%的相对LPIPS;CST*T分别降低56.1%和77.5%的后缀LPIPS,同时相较等待策略分别获得2.73倍和1.69倍像素就绪加速。在HY-WorldPlay协议下,CST*R达到25.66 dB PSNR、0.6902 SSIM和0.1337 LPIPS,与原始推演结果接近。
原文摘要 · Abstract (English)
Chunk-autoregressive video world models typically condition each generated chunk on one action. An action received during sampling must therefore wait for the next chunk, condition future solver evaluations on a state produced under the previous action, or trigger rollback that repeats completed evaluations. We introduce ActionSplice, an inference framework that formulates this problem as Counterfactual State Transport (CST). A lightweight corrector transports the interrupted backbone-native representation toward the matched state induced by the revised action at the same solver step. The world model and sampler remain frozen, and sampling resumes without replaying completed evaluations. The retargeting variant $\mathrm{CST}*{R}$ updates the entire active chunk, while the temporal-splicing variant $\mathrm{CST}*{T}$ preserves a temporal prefix and updates only the suffix. Across minWM-Wan Action2V and HY-WM1.5, $\mathrm{CST}*{R}$ reduces rollback-relative LPIPS by 61.5% and 75.9% relative to direct condition swapping. $\mathrm{CST}*{T}$ reduces suffix LPIPS by 56.1% and 77.5%, respectively, while providing $2.73\times$ and $1.69\times$ pixel-ready speedups over waiting. Under the HY-WorldPlay protocol, $\mathrm{CST}_{R}$ obtains a PSNR of 25.66 dB, an SSIM of 0.6902, and an LPIPS of 0.1337 against the original rollout.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。