arXiv:2510.20206cs.CV2025-10被引 8

通过三阶段优化,让文本生成视频更准更稳

RAPO++: Cross-Stage Prompt Optimization for Text-to-Video Generation via Data Alignment and Test-Time Scaling

  • 用检索和重写增强原始提示,使其更贴合训练数据分布
  • 迭代优化提示,结合空间、时间、运动等多维度反馈提升质量
  • 可适配任意视频生成模型,适合追求高质量输出的研究者

提示词设计在文本到视频(T2V)生成中至关重要,但用户提供的提示通常简短、无结构且与训练数据不匹配,限制了基于扩散模型的T2V生成能力。本文提出跨阶段提示优化框架RAPO++,融合训练数据对齐优化、测试时迭代缩放及大语言模型微调,显著提升生成效果而无需修改生成主干。第一阶段RAPO通过关系图检索语义相关修饰词并重构提示,提升组合性与多对象保真度;第二阶段SSPO引入闭环机制,利用语义对齐、空间保真、时间连贯性及光流等多源反馈迭代优化提示,持续提升视频质量;第三阶段使用SSPO生成的优化提示对重写语言模型进行微调,内化任务优化模式,实现高效高质量提示生成。在五个先进T2V模型和五个基准上的实验证明,RAPO++在语义对齐、组合推理、时间稳定性与物理合理性方面均有显著提升,大幅超越现有方法。结果表明,RAPO++是模型无关、成本低、可扩展的提示优化新标准。代码已开源。

原文摘要 · Abstract (English)

Prompt design plays a crucial role in text-to-video (T2V) generation, yet user-provided prompts are often short, unstructured, and misaligned with training data, limiting the generative potential of diffusion-based T2V models. We present \textbf{RAPO++}, a cross-stage prompt optimization framework that unifies training-data--aligned refinement, test-time iterative scaling, and large language model (LLM) fine-tuning to substantially improve T2V generation without modifying the underlying generative backbone. In \textbf{Stage 1}, Retrieval-Augmented Prompt Optimization (RAPO) enriches user prompts with semantically relevant modifiers retrieved from a relation graph and refactors them to match training distributions, enhancing compositionality and multi-object fidelity. \textbf{Stage 2} introduces Sample-Specific Prompt Optimization (SSPO), a closed-loop mechanism that iteratively refines prompts using multi-source feedback -- including semantic alignment, spatial fidelity, temporal coherence, and task-specific signals such as optical flow -- yielding progressively improved video generation quality. \textbf{Stage 3} leverages optimized prompt pairs from SSPO to fine-tune the rewriter LLM, internalizing task-specific optimization patterns and enabling efficient, high-quality prompt generation even before inference. Extensive experiments across five state-of-the-art T2V models and five benchmarks demonstrate that RAPO++ achieves significant gains in semantic alignment, compositional reasoning, temporal stability, and physical plausibility, outperforming existing methods by large margins. Our results highlight RAPO++ as a model-agnostic, cost-efficient, and scalable solution that sets a new standard for prompt optimization in T2V generation. The code is available at https://github.com/Vchitect/RAPO.

文本生成视频提示优化扩散模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。