arXiv:2607.18724cs.AI2026-07被引 1

针对图文生成提示错误,提出分类型修复分配方案,精准修正语义偏差。

One Rewrite to Fix Them All? Type-Aware Repair Allocation for Text-to-Image Prompt Optimization

论文配图:One Rewrite to Fix Them All? Type-Aware Repair Allocation for Text-to-Image Prompt Optimization
图 1 · 摘自论文原文
  • 按错误类型分配专用修复操作,避免统一扩写导致的语义失真。
  • 在DSG和TIFA数据集上,平均提升5.6和2.6点语义准确率。
  • 无需重训练模型,16秒/提示高效运行,适合快速优化提示词。

文本到图像生成器常因提示词理解错误而产生数量不符、属性错位、关系模糊或文字不可读等问题。提示优化通过重写用户提示来修复这些错误,无需重新训练生成器,已取得良好效果。然而,现有优化器将多种错误统一扩展为单一提示,忽略了不同错误需不同修复语言的本质。本文提出语义提示优化的原子化修复分配框架:每个失败命题被路由至类型条件的修复算子,局部约束再合并为可执行提示。基于此,我们构建了无需训练的类型感知修复分配(TARA)框架,包含诊断、分配、编译与语义修复门控机制——一个精确控制是否采纳单个预设修复的接受或回滚控制器,防止语义退化。在四个冻结生成器上的DSG和TIFA数据集实验表明,TARA在全部八个基准-生成器组合中均达最优语义准确率,较VisualPrompter在DSG和TIFA上分别提升5.6和2.6个百分点,同时保持图像质量,并在相同本地环境下以16.0秒/提示的速度领先于20.0秒/提示。

原文摘要 · Abstract (English)

Text-to-image (T2I) generators often fail to follow their prompts faithfully, producing wrong counts, swapped attributes, ambiguous relations, and illegible text. Prompt optimization repairs such failures by rewriting the user prompt, requiring no generator retraining, and has yielded promising results. However, existing optimizers absorb heterogeneous failures into one uniform prompt expansion, even though each calls for different repair language. We formulate semantic prompt optimization as atomic repair allocation: each failed proposition is routed to a type-conditioned repair operator before the resulting local constraints are compiled into one executable prompt. We instantiate this formulation in the training-free Type-Aware Repair Allocation (TARA) framework, which separates diagnosis, allocation, compilation, and a semantic repair gate, an accept-or-revert controller over exactly one prescribed repair that prevents semantic regressions. Extensive experiments on DSG and TIFA across four frozen generators demonstrate that TARA achieves the best semantic accuracy in all eight benchmark-generator cells, improving over VisualPrompter by 5.6 and 2.6 points on DSG and TIFA, respectively, while maintaining image quality and running fastest in our matched local setting at 16.0 seconds versus 20.0 seconds per prompt.

提示优化图像生成语义修复无训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。