arXiv:2606.28016cs.CV2026-06

用强化学习让视频生成更懂时间顺序,避免错乱和延迟。

TempAct: Advancing Temporal Plausibility in Autoregressive Video Generation via Planner-Executor RL

论文配图:TempAct: Advancing Temporal Plausibility in Autoregressive Video Generation via Planner-Executor RL
图 1 · 摘自论文原文
  • 分步规划+执行的强化学习框架,明确每一步该生成什么。
  • 在Self-Forcing和LongLive数据集上提升时间一致性,视觉质量不下降。
  • 适合需要精准时序控制的视频生成任务,如动画、动作预测。

自回归视频扩散模型通过逐块生成实现低延迟流式输出,但时间指令跟随模糊:单一全局提示无法指定每块内容,而逐步切换提示常导致反应延迟、语义混淆和错误传播。监督微调存在暴露偏差,基于回放的蒸馏仍优化底层去噪或教师分布匹配,无法直接约束动作顺序与提示切换正确性。本文提出TempAct,一种规划-执行强化学习框架,联合优化时间分解与步骤条件执行。利用LLM规划器生成可执行的分段提示,训练自回归扩散执行器在自身生成历史下遵循这些提示。核心机制为层级组探索:候选计划形成规划组,每计划生成多个延续构成执行组,共享视觉上下文,实现长程结果的计划级奖励分配与提示切换行为的执行级奖励分配。设计分层奖励:规划器获得计划质量与全程时间反馈,执行器获得局部过渡奖励、美学正则化与KL约束。实验表明,TempAct在Self-Forcing和LongLive数据集上显著提升时间一致性,同时保持整体视觉质量。

原文摘要 · Abstract (English)

Autoregressive (AR) video diffusion models enable low-latency streaming generation by synthesizing videos chunk by chunk with cached visual context, but this chunk-wise formulation makes temporal instruction following ambiguous. A single global prompt does not specify which sub-event should be realized in each chunk, while naively switching to step-wise prompts often leads to delayed reactions, blended step semantics, and error propagation across prompt transitions. These failures are difficult to address with supervised fine-tuning or distillation alone: SFT suffers from exposure bias, while rollout-based distillation still optimizes low-level denoising or teacher-distribution matching rather than directly enforcing action ordering and prompt-transition correctness. We address these challenges with TempAct, a planner--executor reinforcement learning framework that jointly optimizes temporal decomposition and step-conditioned execution for temporally plausible AR video generation. TempAct uses an LLM planner to explore span-aware step prompts that are executable by the video model, and trains an AR diffusion executor to follow these prompts under its own generated histories. Its key mechanism is hierarchical group exploration: candidate plans form planning groups, and each plan induces an execution group of multiple continuations from a shared visual context, enabling plan-level credit assignment for long-horizon temporal outcomes and executor-level credit assignment for prompt-switch behavior. We further design hierarchical rewards that combine plan-quality and full-video temporal feedback for the planner with local transition-level step-following rewards, aesthetic regularization, and KL constraints for the executor. Experiments on Self-Forcing and LongLive show that TempAct improves temporal consistency while preserving overall visual quality.

视频生成强化学习时序控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。