arXiv:2608.21713cs.CVcs.CL2026-08中稿 · ECCV

发现图像生成失败主因在规划阶段,修复计划即可提升效果

The Plan, Not the Decoder: Diagnosing and Repairing Compositional Failure in Reasoning-Augmented Text-to-Image Generation

论文配图:The Plan, Not the Decoder: Diagnosing and Repairing Compositional Failure in Reasoning-Augmented Text-to-Image Generation
图 1 · 摘自论文原文
  • 通过可编辑的文本计划分离诊断规划与解码问题
  • 94%布局准确率说明解码器忠实执行计划,规划才是瓶颈
  • 仅修改计划几何信息就能提升生成质量10.7分,无需重训练

如GoT-R1等推理增强型图文生成模型会在生成图像标记前输出包含对象名、属性和边界框的显式文本计划。当模型对组合提示产生错误时,是计划出错,还是解码器不忠实?由于计划为机器可读,可在解码前编辑,使两者可分离。实验验证:在模型内部交换两个边界框,导致检测器准确率从0.75降至0.48(p<1e-3),而基于VQA的空间度量上升。五名人类评估者中81%与检测器一致,57%与VQA判断一致。因此采用几何评分。在可靠测量下,解码器为忠实执行者:94%生成布局实现计划关系,且对象-边界框绑定在计划重排后仍保持。规划器才是瓶颈:因措辞依赖导致错误——同一布局中“左”正确率98%,“右”仅54%,并发现栅格顺序偏差;杂乱几何被解码器忠实复现。修复计划即可改进图像而无需重训练:符号验证+5.0分(p<1e-3),最小原位修复+6.0(p=0.02),仅重写边界框几何+10.7(p<1e-4),完全替换计划+13.3(p=1e-4)。增益与计划文风及规划器概率无关,但与几何相关。模块化规划-解码设计可行,前提是计划内部一致:盒-文矛盾会导致对象重复与身份融合。论文发布计划一致性评估协议、所有计划及12,000张生成图像。

原文摘要 · Abstract (English)

Reasoning-augmented text-to-image models such as GoT-R1 emit an explicit textual plan - object names, attributes, and bounding boxes - before generating image tokens. When such a model fails a compositional prompt, is the plan wrong, or is the plan right and the decoder unfaithful? Because the plan is machine-readable it can be edited before decoding, which makes the two separable. We first validate the ruler. Swapping the two bounding boxes inside the model's own chain demonstrably flips the generated layout: detector-based accuracy falls 0.75 -> 0.48 (p<1e-3), while a widely used VQA-based spatial metric rises. A five-rater human study agrees with the detector on 81% of items and with the VQA judge on 57%. All spatial results therefore use geometric scoring. Under sound measurement the decoder is a faithful executor: 94% of generated layouts realize the planned relation, and object-box binding survives reordering of the plan's object segments. The planner is the bottleneck. It writes wrong relations for phrasing-dependent reasons - 98% accuracy on "left" against 54% on "right" for semantically identical layouts, a raster-order bias we isolate with a mention-order control - and cluttered geometry that the decoder faithfully reproduces. Editing the plan therefore fixes the image without retraining: symbolic verification with resampling gives +5.0 points (p<1e-3), minimal in-place repair +6.0 (p=.02), rewriting only box geometry +10.7 (p<1e-4), and replacing the plan outright +13.3 (p=1e-4). Gains are indifferent to the plan's prose style and to its likelihood under the planner, but not to its geometry. Modular planner-decoder designs are therefore viable, provided the plan is internally consistent: box-text contradictions induce object duplication and identity fusion. We release the plan-fidelity evaluation protocol, all plans, and 12k generated images.

图文生成规划修复视觉推理模型诊断

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。