无需训练即可实现多参考图像的布局感知合成,保持身份一致与背景连贯。
LAMIC: Layout-Aware Multi-Image Composition via Scalability of Multimodal Diffusion Transformer
- 引入两种可插拔注意力机制,提升实体分离与布局感知能力。
- 在多个指标上超越现有方法,尤其在复杂组合任务中表现最优。
- 零样本泛化强,适合需要快速生成多图一致内容的场景。
在可控图像生成中,从多个参考图生成具空间布局感知且一致的图像仍是开放挑战。本文提出LAMIC,首个以无训练方式将单参考扩散模型扩展至多参考场景的布局感知多图合成框架。基于MMDiT模型,LAMIC引入两种即插即用注意力机制:1)分组隔离注意力(GIA)增强实体解耦;2)区域调制注意力(RMA)实现布局感知生成。为全面评估模型能力,进一步提出三项新指标:1)包含率(IN-R)与填充率(FI-R)用于评估布局控制;2)背景相似度(BG-S)衡量背景一致性。大量实验表明,LAMIC在多数主要指标上达到当前最优性能:在所有设置下持续优于现有多参考基线,在ID-S、BG-S、IN-R和AVG得分上均领先,且在复杂组合任务中取得最佳DPG。结果证明其在身份保持、背景保留、布局控制和提示遵循方面的卓越能力,且全程无需训练或微调,展现出强大的零样本泛化能力。通过继承先进单参考模型的优势并实现无缝扩展至多图场景,LAMIC确立了可控多图合成的新范式。随着基础模型演进,LAMIC性能有望随之提升。代码已开源:https://github.com/Suchenl/LAMIC。
原文摘要 · Abstract (English)
In controllable image synthesis, generating coherent and consistent images from multiple references with spatial layout awareness remains an open challenge. We present LAMIC, a Layout-Aware Multi-Image Composition framework that, for the first time, extends single-reference diffusion models to multi-reference scenarios in a training-free manner. Built upon the MMDiT model, LAMIC introduces two plug-and-play attention mechanisms: 1) Group Isolation Attention (GIA) to enhance entity disentanglement; and 2) Region-Modulated Attention (RMA) to enable layout-aware generation. To comprehensively evaluate model capabilities, we further introduce three metrics: 1) Inclusion Ratio (IN-R) and Fill Ratio (FI-R) for assessing layout control; and 2) Background Similarity (BG-S) for measuring background consistency. Extensive experiments show that LAMIC achieves state-of-the-art performance across most major metrics: it consistently outperforms existing multi-reference baselines in ID-S, BG-S, IN-R and AVG scores across all settings, and achieves the best DPG in complex composition tasks. These results demonstrate LAMIC's superior abilities in identity keeping, background preservation, layout control, and prompt-following, all achieved without any training or fine-tuning, showcasing strong zero-shot generalization ability. By inheriting the strengths of advanced single-reference models and enabling seamless extension to multi-image scenarios, LAMIC establishes a new training-free paradigm for controllable multi-image composition. As foundation models continue to evolve, LAMIC's performance is expected to scale accordingly. Our implementation is available at: https://github.com/Suchenl/LAMIC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。