让生成提示词随推理过程动态调整,提升图文生成精准度
Rethinking Prompt Design for Inference-time Scaling in Text-to-Visual Generation
- 推理时动态重写提示词,根据生成结果反馈优化
- 在VBench 2.0上实现15%性能提升,突破固定提示瓶颈
- 适合追求高精度图文生成的开发者与研究者
文本到视觉生成中,用户意图与生成结果间的精准对齐仍是核心挑战,单次尝试常无法产出理想输出。以往方法主要通过增加采样步数或种子数来扩展生成过程,但很快达到质量上限,根源在于提示词保持不变。为此,本文提出推理时提示词重设计框架PRIS,通过分析生成图像中的重复失败模式,动态重构提示词并重新生成。为提供细粒度对齐反馈,引入元素级事实修正验证器,相比整体评估更准确可解释。在文本到图像与文本到视频基准测试中均验证有效,尤其在VBench 2.0上提升15%。结果表明,联合扩大提示词与视觉生成规模,是充分挖掘推理阶段缩放定律的关键。
原文摘要 · Abstract (English)
Achieving precise alignment between user intent and generated visuals remains a central challenge in text-to-visual generation, as a single attempt often fails to produce the desired output. To handle this, prior approaches mainly scale the visual generation process (e.g., increasing sampling steps or seeds), but this quickly leads to a quality plateau. This limitation arises because the prompt, crucial for guiding generation, is kept fixed. To address this, we propose Prompt Redesign for Inference-time Scaling, coined PRIS, a framework that adaptively revises the prompt during inference in response to the scaled visual generations. The core idea of PRIS is to review the generated visuals, identify recurring failure patterns across visuals, and redesign the prompt accordingly before regenerating the visuals with the revised prompt. To provide precise alignment feedback for prompt revision, we introduce a new verifier, element-level factual correction, which evaluates the alignment between prompt attributes and generated visuals at a fine-grained level, achieving more accurate and interpretable assessments than holistic measures. Extensive experiments on both text-to-image and text-to-video benchmarks demonstrate the effectiveness of our approach, including a 15% gain on VBench 2.0. These results highlight that jointly scaling prompts and visuals is key to fully leveraging scaling laws at inference-time. Visualizations are available at the website: https://subin-kim-cv.github.io/PRIS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。