arXiv:2509.26644cs.CVcs.AI2025-09被引 3

无需训练即可精准控制图像中物体位置,提升生成准确性。

Stitch: Training-Free Position Control in Multimodal Diffusion Transformers

  • 通过自动生成边界框,分块生成并拼接物体实现位置控制。
  • 在PosEval上使FLUX模型位置任务性能提升218%,超越现有方法54%。
  • 适用于主流扩散模型,无需微调,适合需要精确布局的生成场景。

文本到图像生成模型近年发展迅速,但准确捕捉“上方”或“右侧”等空间关系仍具挑战。早期方法依赖外部位置控制,但随架构演进而难以适配现代模型。本文提出Stitch,一种无需训练的方法,通过自动生成边界框,将多模态扩散变换器(MMDiT)中的物体分块生成并无缝拼接,实现空间精准控制。研究发现,特定注意力头可在生成中途提取并裁剪单个物体,无需完成整图。在作者构建的PosEval基准测试中,该任务扩展了基本生成评估,涵盖五个新任务,揭示顶级模型在位置生成上仍有巨大提升空间。在Qwen-Image、FLUX和SD3.5上测试,Stitch均显著增强基线模型,使FLUX在GenEval位置任务上提升218%,在PosEval上提升206%;在PosEval上以领先54%的成绩达成当前最优表现。所有改进均在无训练条件下实现,代码已开源。

原文摘要 · Abstract (English)

Text-to-Image (T2I) generation models have advanced rapidly in recent years, but accurately capturing spatial relationships like "above" or "to the right of" poses a persistent challenge. Earlier methods improved spatial relationship following with external position control. However, as architectures evolved to enhance image quality, these techniques became incompatible with modern models. We propose Stitch, a training-free method for incorporating external position control into Multi-Modal Diffusion Transformers (MMDiT) via automatically-generated bounding boxes. Stitch produces images that are both spatially accurate and visually appealing by generating individual objects within designated bounding boxes and seamlessly stitching them together. We find that targeted attention heads capture the information necessary to isolate and cut out individual objects mid-generation, without needing to fully complete the image. We evaluate Stitch on PosEval, our benchmark for position-based T2I generation. Featuring five new tasks that extend the concept of Position beyond the basic GenEval task, PosEval demonstrates that even top models still have significant room for improvement in position-based generation. Tested on Qwen-Image, FLUX, and SD3.5, Stitch consistently enhances base models, even improving FLUX by 218% on GenEval's Position task and by 206% on PosEval. Stitch achieves state-of-the-art results with Qwen-Image on PosEval, improving over previous models by 54%, all accomplished while integrating position control into leading models training-free. Code is available at https://github.com/ExplainableML/Stitch.

图像生成扩散模型位置控制零样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。