通过测试时噪声优化,提升文本到视频生成的细粒度对齐能力。
NoisEasier: Test-Time Noise Optimization for Text-to-Video Generation

- 在推理阶段通过可微分噪声优化,无需修改模型即可改进生成效果。
- 在多个基准上实现超过10%的提升,尤其在属性绑定和物体交互上表现突出。
- 适合需要高精度控制但无法重新训练模型的场景,如快速部署与调试。
扩散模型最近推动了文本到视频(T2V)生成的发展,但仍面临细粒度组合对齐难题,如属性绑定、空间关系和物体交互。尽管基于奖励的微调能提升对齐性,但易受奖励欺骗影响,且对新提示分布适应性差。本文提出 NoisEasier,一种测试时缩放框架,通过可微分奖励引导的噪声优化,在不修改底层模型的前提下提升 T2V 生成质量。结合高效短步生成器与多目标奖励设计,NoisEasier 在实际推理预算下实现稳定且实用的测试时优化。关键洞察在于联合优化整个随机轨迹,相比仅优化初始隐变量,能加速奖励收敛并显著改善组合对齐,计算开销几乎不变。在 VBench 与 T2V-CompBench 上的实验表明,该方法在多种骨干模型上均取得一致改进,在属性绑定、物体交互和数理理解等挑战维度平均提升超10%。总体而言,NoisEasier 为基于奖励的微调提供灵活替代与互补增强,确立测试时缩放在可控文本到视频生成中的有效性。
原文摘要 · Abstract (English)
Diffusion models have recently advanced text-to-video (T2V) generation, yet they still struggle with fine-grained compositional alignment, such as attribute binding, spatial relations, and object interactions. While reward-based fine-tuning improves alignment, it is susceptible to reward hacking and adapts poorly to new prompt distributions. In this work, we propose NoisEasier, a test-time scaling framework that improves T2V generation through differentiable reward-guided noise optimization without modifying the underlying model. By combining efficient short-step generators with a multi-objective reward formulation, NoisEasier enables stable and practical test-time optimization under realistic inference budgets. Our key insight is that jointly optimizing the entire stochastic trajectory accelerates reward convergence and improves compositional alignment over optimizing only the initial latent, with negligible additional computational and time cost. Experiments on VBench and T2V-CompBench demonstrate consistent improvements across multiple backbones, achieving over 10% average gains on challenging dimensions such as attribute binding, object interaction, and numeracy. Overall, NoisEasier serves as both a flexible alternative and a complementary enhancement to reward-based fine-tuning, establishing test-time scaling as an effective paradigm for controllable text-to-video generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。