用强化学习自动优化提示词,让文本生成视频更符合物理规律。
PhyPrompt: RL-based Prompt Refinement for Physically Plausible Text-to-Video Generation
- 两阶段强化学习框架,先学物理常识再优化提示词
- 在VideoPhy2上物理合理性提升11个百分点至66.8%
- 小模型胜过大模型,适合需要真实物理感的生成任务
当前顶尖的文本到视频生成模型虽视觉质量高,但常违背物理规律。我们发现根源在于提示词缺乏物理约束,而非模型能力不足:手动添加物理细节可提升真实性,但需专业知识且难扩展。为此提出PhyPrompt,一个基于强化学习的两阶段提示词优化框架。首先,在聚焦物理的思维链数据集上微调大语言模型,融合物体运动与受力交互等原理,同时保留用户意图;其次,采用分阶段奖励策略,初期侧重语义一致性,后期逐步转向物理常识。该策略实现协同优化:PhyPrompt-7B在VideoPhy2上联合成功率达40.8%(提升8.6个百分点),物理常识提升11个百分点(从55.8%增至66.8%),语义一致率提升4.4个百分点(达47.8%)。显著优于单目标训练,展现出超越传统多目标权衡的组合式提示发现能力。其性能超过GPT-4o(+3.8%联合成功率)和DeepSeek-V3(+2.2%,参数量大100倍),仅用70亿参数即达成。该方法零样本迁移至多种T2V架构(Lavie、VideoCrafter2、CogVideoX-5B),最高提升16.8%,表明领域专用强化学习结合组合式奖励曲线,优于通用规模扩展。
原文摘要 · Abstract (English)
State-of-the-art text-to-video (T2V) generators frequently violate physical laws despite high visual quality. We show this stems from insufficient physical constraints in prompts rather than model limitations: manually adding physics details reliably produces physically plausible videos, but requires expertise and does not scale. We present PhyPrompt, a two-stage reinforcement learning framework that automatically refines prompts for physically realistic generation. First, we fine-tune a large language model on a physics-focused Chain-of-Thought dataset to integrate principles like object motion and force interactions while preserving user intent. Second, we apply Group Relative Policy Optimization with a dynamic reward curriculum that initially prioritizes semantic fidelity, then progressively shifts toward physical commonsense. This curriculum achieves synergistic optimization: PhyPrompt-7B reaches 40.8\% joint success on VideoPhy2 (8.6pp gain), improving physical commonsense by 11pp (55.8\% to 66.8\%) while simultaneously increasing semantic adherence by 4.4pp (43.4\% to 47.8\%). Remarkably, our curriculum exceeds single-objective training on both metrics, demonstrating compositional prompt discovery beyond conventional multi-objective trade-offs. PhyPrompt outperforms GPT-4o (+3.8\% joint) and DeepSeek-V3 (+2.2\%, 100$\times$ larger) using only 7B parameters. The approach transfers zero-shot across diverse T2V architectures (Lavie, VideoCrafter2, CogVideoX-5B) with up to 16.8\% improvement, establishing that domain-specialized reinforcement learning with compositional curricula surpasses general-purpose scaling for physics-aware generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。