轻量级提示增强框架,让小模型也能精准优化图像生成指令。
APE: Agentic Prompt Enhancer for Image Generation and Editing

- 用小模型后训练成提示增强代理,可单轮重写或分角色协作优化
- 在复杂组合任务上超越基础模型,接近闭源大模型效果
- 适合需要低成本、高可控性的图像生成与编辑场景
自然语言已成为图像生成与编辑的强大接口,但文本引导的视觉系统对提示表述极为敏感。语义相似的请求因措辞、细节程度和视觉约束明确性不同,会产生不同输出,促使将提示增强作为可训练组件而非用户随意选择。现有强增强器常依赖如 ChatGPT 或 Gemini 等大型专有大模型,增加成本、延迟和部署依赖。我们提出轻量级框架 Agentic Prompt Enhancer (APE),通过后训练小语言模型(SLMs)作为提示增强代理。APE 支持单代理重写与角色专业化多代理增强。其单代理版本 SAPE 一次性重写提示;多代理版本 MAPE 将增强分解为路由-重写-组合流程,有效处理对象、属性、空间关系及编辑的复合约束。结合任务感知奖励与后训练协议,APE 提升视觉对齐与提示遵循能力,无需修改下游视觉模型。在具有挑战性的图像生成与编辑基准测试中,后训练的小提示增强器稳定优于基线模型,缩小与闭源增强器的差距;尤其在复杂组合任务中,MAPE 表现突出。
原文摘要 · Abstract (English)
Natural language has become a powerful interface for image generation and editing, yet text-guided visual systems remain highly sensitive to prompt formulation. Semantically similar requests can produce different outputs depending on wording, specificity, and how explicitly visual constraints are stated, motivating prompt enhancement as a trainable component rather than a peripheral user choice. Existing strong enhancers often rely on large, proprietary LLMs such as ChatGPT or Gemini, adding cost, latency, and deployment dependence to the visual generation pipeline. We propose Agentic Prompt Enhancer (APE), a lightweight framework that post-trains small language models (SLMs) as prompt-enhancement agents. APE supports both single-agent rewriting and role-specialized multi-agent enhancement. Its single-agent instantiation, SAPE, rewrites the prompt in one pass, while its multi-agent instantiation, MAPE, decomposes enhancement into a router--rewriter--composer process for handling compositional constraints over objects, attributes, spatial relations, and edits. With task-aware rewards and post-training protocols, APE improves visual alignment and prompt following without modifying the downstream visual model. Experiments on challenging image generation and editing benchmarks demonstrate that post-trained small prompt enhancers reliably outperform their base counterparts, narrowing the gap to closed-source prompt enhancers; in addition, MAPE proves particularly strong on complex compositional tasks within these benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。