arXiv:2601.18543cs.CV2026-01被引 13

让文字生成图像更智能:用智能体自主迭代优化结果。

GenAgent: Scaling Text-to-Image Generation via Agentic Multimodal Reasoning

  • 用智能体框架分离理解与生成,动态调用图像模型工具
  • 在GenEval++上提升23.6%,WISE上提升14%
  • 支持多轮交互、跨工具泛化,适合复杂图像生成任务

我们提出GenAgent,通过智能体多模态推理统一视觉理解与生成。不同于需高昂训练成本且存在理解-生成权衡的统一模型,GenAgent采用解耦架构:视觉理解由多模态模型完成,生成则通过将图像生成模型作为可调用工具。关键在于,相比静态流水线的模块化系统,该设计支持自主多轮交互,智能体能生成包含推理、工具调用、判断与反思的多模态思维链,持续优化输出。采用两阶段训练策略:首先基于高质量工具调用与反思数据进行监督微调以启动行为;其次采用端到端智能体强化学习,结合点状奖励(最终图像质量)与成对奖励(反思准确性),并引入轨迹重采样以增强多轮探索能力。GenAgent显著提升基础生成器FLUX.1-dev在GenEval++上的表现(+23.6%)和WISE上的表现(+14%)。除性能提升外,本框架还展现出三项关键特性:1)跨工具泛化至不同能力的生成器;2)测试时可扩展,交互轮次越多效果越佳;3)任务自适应推理,能自动适配不同任务需求。代码将开源。

原文摘要 · Abstract (English)

We introduce GenAgent, unifying visual understanding and generation through an agentic multimodal model. Unlike unified models that face expensive training costs and understanding-generation trade-offs, GenAgent decouples these capabilities through an agentic framework: understanding is handled by the multimodal model itself, while generation is achieved by treating image generation models as invokable tools. Crucially, unlike existing modular systems constrained by static pipelines, this design enables autonomous multi-turn interactions where the agent generates multimodal chains-of-thought encompassing reasoning, tool invocation, judgment, and reflection to iteratively refine outputs. We employ a two-stage training strategy: first, cold-start with supervised fine-tuning on high-quality tool invocation and reflection data to bootstrap agent behaviors; second, end-to-end agentic reinforcement learning combining pointwise rewards (final image quality) and pairwise rewards (reflection accuracy), with trajectory resampling for enhanced multi-turn exploration. GenAgent significantly boosts base generator(FLUX.1-dev) performance on GenEval++ (+23.6\%) and WISE (+14\%). Beyond performance gains, our framework demonstrates three key properties: 1) cross-tool generalization to generators with varying capabilities, 2) test-time scaling with consistent improvements across interaction rounds, and 3) task-adaptive reasoning that automatically adjusts to different tasks. Our code will be available at \href{https://github.com/deep-kaixun/GenAgent}{this url}.

文本生成图像智能体多模态推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。