arXiv:2602.18282cs.CV2026-02AAAI

让复杂描述精准生成多个带细节的物体实例

DEIG: Detail-Enhanced Instance Generation with Fine-Grained Semantic Control

论文配图:DEIG: Detail-Enhanced Instance Generation with Fine-Grained Semantic Control
图 1 · 摘自论文原文
  • 用细粒度语义提取与掩码注意力防属性串扰
  • 在多实例生成中实现更高空间一致性和语义准确率
  • 适合需要精确控制物体细节的视觉生成任务

多实例生成在空间布局和属性绑定方面已取得显著进展,但面对复杂文本描述时仍存在细粒度语义理解不足的问题。为此,我们提出DEIG框架,通过实例细节提取器(IDE)将文本编码器嵌入转化为紧凑的实例感知表示,并引入基于实例的掩码注意力机制的细节融合模块(DFM),有效防止属性跨实例泄露。该框架能生成与丰富、局部化文本描述高度匹配的视觉一致多实例场景。为支持细粒度监督,我们构建了一个高质量数据集,使用视觉语言模型生成详细的组合式实例标注。同时,我们提出DEIG-Bench基准,包含区域级注释和多属性提示,适用于人与物的评估。实验表明,DEIG在多个基准上持续优于现有方法,在空间一致性、语义准确性和组合泛化能力方面表现优异。此外,DEIG可作为即插即用模块,轻松集成至标准扩散生成流程。

原文摘要 · Abstract (English)

Multi-Instance Generation has advanced significantly in spatial placement and attribute binding. However, existing approaches still face challenges in fine-grained semantic understanding, particularly when dealing with complex textual descriptions. To overcome these limitations, we propose DEIG, a novel framework for fine-grained and controllable multi-instance generation. DEIG integrates an Instance Detail Extractor (IDE) that transforms text encoder embeddings into compact, instance-aware representations, and a Detail Fusion Module (DFM) that applies instance-based masked attention to prevent attribute leakage across instances. These components enable DEIG to generate visually coherent multi-instance scenes that precisely match rich, localized textual descriptions. To support fine-grained supervision, we construct a high-quality dataset with detailed, compositional instance captions generated by VLMs. We also introduce DEIG-Bench, a new benchmark with region-level annotations and multi-attribute prompts for both humans and objects. Experiments demonstrate that DEIG consistently outperforms existing approaches across multiple benchmarks in spatial consistency, semantic accuracy, and compositional generalization. Moreover, DEIG functions as a plug-and-play module, making it easily integrable into standard diffusion-based pipelines.

多实例生成细粒度控制扩散模型视觉语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。