arXiv:2512.20362cs.CV2025-12被引 2

让AI画画时像人一样一步步检查修正,提升准确性且不增加计算成本。

CRAFT: Continuous Reasoning and Agentic Feedback Tuning for Multimodal Text-to-Image Generation

  • 将用户指令转为可验证的视觉约束,逐条检查图像是否达标
  • 仅在违反约束时才修改提示,迭代优化后准确率显著提升
  • 适合需要高精度生成的轻量级模型,推理开销几乎为零

近期研究显示,推理阶段的思考与反思能提升文本到图像生成效果而无需重新训练。然而,现有方法多依赖隐式、整体性的批评或无约束的提示重写,导致行为难以解释、控制或终止。相比之下,大语言模型通过显式、结构化的思考(如验证、针对性修正和早期停止)获益良多。我们提出CRAFT(连续推理与代理反馈微调),一种无需训练、对模型无依赖的多模态图像生成框架。CRAFT将用户提示转化为一组显式、依赖结构化的视觉约束,利用视觉-语言模型验证生成图像,并在特定约束被违反时进行目标性提示更新。该迭代过程包含明确的停止条件,形成可解释且可控的推理期优化循环。在多个模型族和挑战性基准上,CRAFT持续提升组合准确性、文字渲染质量和偏好评估表现,尤其对轻量级生成器提升显著。重要的是,这些改进带来的推理时间开销可忽略不计,使小型或低成本模型逼近昂贵系统的质量。结果表明,显式结构化、约束驱动的推理期思考是提升多模态生成模型可靠性的关键。

原文摘要 · Abstract (English)

Recent work has shown that inference-time reasoning and reflection can improve text-to-image generation without retraining. However, existing approaches often rely on implicit, holistic critiques or unconstrained prompt rewrites, making their behavior difficult to interpret, control, or stop reliably. In contrast, large language models have benefited from explicit, structured forms of **thinking** based on verification, targeted correction, and early stopping. We introduce CRAFT (Continuous Reasoning and Agentic Feedback Tuning), a training-free and model-agnostic framework for multimodal image generation. CRAFT transforms a user prompt into a set of explicit, dependency-structured visual constraints, verifies generated images using a vision-language model, and performs targeted prompt updates only when specific constraints are violated. This iterative process includes an explicit stopping criterion, resulting in an interpretable and controllable inference-time refinement loop. Across multiple model families and challenging benchmarks, CRAFT consistently improves compositional accuracy, text rendering, and preference-based evaluations, with particularly strong gains for lightweight generators. Importantly, these improvements incur only a negligible inference-time overhead, allowing smaller or cheaper models to approach the quality of substantially more expensive systems. Our results suggest that explicitly structured, constraint-driven inference-time reasoning is a key ingredient for improving the reliability of multimodal generative models.

图像生成推理优化提示工程轻量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。