arXiv:2603.08403cs.CV2026-03被引 1

让视频生成能按复杂指令持续执行动作,避免跑偏和乱序。

SPIRAL: Self-Evolving Action-Conditioned Video Generation via Reflective Planning Agents

  • 用计划-执行-反思闭环,分步生成并修正视频片段。
  • 在多个基准上显著提升长时视频的动作准确性和连贯性。
  • 适合需要精确动作序列生成的场景,如教学演示或机器人训练。

长时程动作条件视频生成旨在合成遵循复杂动作指令的时序连贯视频,要求动作顺序合理、持续执行且场景一致,超越传统文本到视频(TI2V)模型的短期保真度。现有单次生成模型多为开环运行,导致动作执行不完整、运动幻觉及时间漂移。为此,我们提出SPIRAL,一种闭环框架,通过递进规划与迭代反思实现长时程动作条件视频生成。具体而言,SPIRAL构建思考-行动-反思流程:计划代理(PlanAgent)将高层目标分解为子动作,指导视频生成器(VideoGenerator)结合记忆上下文生成每一段视频;评估代理(CriticAgent)则对中间视频段进行评价,提供修正反馈以实现迭代优化。该闭环设计还支持自演化:利用计划代理提出的动作与评估代理生成的奖励,基于GRPO方法对视频生成器进行后训练,增强其长时一致性。此外,我们引入了任务特定的数据集ActVideoGen-Dataset,并建立了专用评估基准ActVideoGen-Bench,用于衡量动作质量与时序连贯性。在多个TI2V骨干模型上,结合自演化策略的实验表明,SPIRAL在ActVideoGen-Bench和VBench上均取得一致提升,验证了其有效性。

原文摘要 · Abstract (English)

Long-horizon action-conditioned video generation aims to synthesize temporally coherent videos that follow complex action instructions over extended horizons, requiring procedural ordering, persistent action execution, and scene consistency beyond conventional TI2V's short-term fidelity. Existing single-shot video generation models typically operate in an open-loop manner, leading to incomplete action execution, hallucinated motions, and temporal drift. To address this, we propose SPIRAL, a closed-loop framework that performs sequential planning and iterative reflection for action-conditioned long-horizon video generation. Specifically, SPIRAL instantiates a think-act-reflect process: a PlanAgent decomposes high-level goals into sub-actions, which condition a VideoGenerator to synthesize each segment alongside a memory context, while a CriticAgent evaluates intermediate video segments to provide corrective feedback for iterative refinement. This closed-loop design further supports self-evolution by utilizing PlanAgent-proposed actions and CriticAgent-derived rewards for GRPO-based post-training to enhance the video generator's long-horizon consistency. Moreover, we introduce ActVideoGen-Dataset for task-specific training, and establish ActVideoGen-Bench as a dedicated evaluation suite for measuring action quality and temporal coherence. Experiments across multiple TI2V backbones alongside the self-evolving strategy show consistent gains on ActVideoGen-Bench and VBench, demonstrating the effectiveness of SPIRAL.

视频生成动作控制闭环生成自演化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。