用提示词自动优化动态视觉效果生成,无需修改模型。
P-Flow: Prompting Visual Effects Generation
- 通过视觉语言模型在生成时迭代优化提示词
- 实现高保真、多样化的爆炸等动态效果定制
- 适合需要精细控制视觉特效的创作者
视频生成模型虽已能较好理解文本提示,但对动态视觉效果(如物体粉碎、爆炸等)的定制仍不充分。以往方法多关注主体或摄像机的低层运动控制,依赖轨迹等显式信号;而动态视觉效果具有高层语义,更适于通过文本提示控制。然而,人工编写精准描述此类效果的提示极为困难,需复杂的时间推理与反复调整。为此,我们提出P-Flow——一种无需训练的动态视觉效果定制框架,不修改底层模型。它利用视觉语言模型的语义与时间推理能力,在测试阶段进行提示词优化,根据参考视频与生成结果的差异,迭代改进提示词,使其在新场景中更准确地诱发目标动态效果。实验表明,P-Flow在文本到视频及图像到视频任务中均优于现有方法,生成效果保真度高且多样性好。代码已开源。
原文摘要 · Abstract (English)
Recent advancements in video generation models have significantly improved their ability to follow text prompts. However, the customization of dynamic visual effects, defined as temporally evolving and appearance-driven visual phenomena like object crushing or explosion, remains underexplored. Prior works on motion customization or control mainly focus on low-level motions of the subject or camera, which can be guided using explicit control signals such as motion trajectories. In contrast, dynamic visual effects involve higher-level semantics that are more naturally suited for control via text prompts. However, it is hard and time-consuming for humans to craft a single prompt that accurately specifies these effects, as they require complex temporal reasoning and iterative refinement over time. To address this challenge, we propose P-Flow, a novel training-free framework for customizing dynamic visual effects in video generation without modifying the underlying model. By leveraging the semantic and temporal reasoning capabilities of vision-language models, P-Flow performs test-time prompt optimization, refining prompts based on the discrepancy between the visual effects of the reference video and the generated output. Through iterative refinement, the prompts evolve to better induce the desired dynamic effect in novel scenes. Experiments demonstrate that P-Flow achieves high-fidelity and diverse visual effect customization and outperforms other models on both text-to-video and image-to-video generation tasks. Code is available at https://github.com/showlab/P-Flow.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。