从一张图生成多个语义清晰的实例，解决相似物体混淆问题。
MIFO: Learning and Synthesizing Multi-Instance from One Image
- 用惩罚性注意力优化分离相似语义，提升学习精度。
- 引入框控注意力机制，合成时精准控制布局并减少语义泄露。
- 适合处理相似外观或罕见物体的多实例生成任务。
本文提出一种从单张图像中精确学习与合成多实例语义的方法。该问题难点在于训练数据有限，尤其当待学习实例语义或外观相近时更为困难。为此，我们设计基于惩罚的注意力优化，在学习阶段解耦相似语义；在合成阶段，引入并优化注意力层中的框控机制，进一步缓解语义泄露，同时精确控制输出布局。实验表明，该方法在保持高生成质量的同时，显著平衡了可编辑性与实例一致性。即使面对语义或视觉上相似的实例或少见物体，仍具备强鲁棒性。代码已公开于 https://github.com/Kareneveve/MIFO。
原文摘要 · Abstract (English)
This paper proposes a method for precise learning and synthesizing multi-instance semantics from a single image. The difficulty of this problem lies in the limited training data, and it becomes even more challenging when the instances to be learned have similar semantics or appearance. To address this, we propose a penalty-based attention optimization to disentangle similar semantics during the learning stage. Then, in the synthesis, we introduce and optimize box control in attention layers to further mitigate semantic leakage while precisely controlling the output layout. Experimental results demonstrate that our method achieves disentangled and high-quality semantic learning and synthesis, strikingly balancing editability and instance consistency. Our method remains robust when dealing with semantically or visually similar instances or rare-seen objects. The code is publicly available at https://github.com/Kareneveve/MIFO
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。