实现多概念3D内容生成，精准区分每个概念的属性与位置。
MultiDreamer3D: Multi-concept 3D Customization with Concept-Aware Diffusion Guidance
- 分步生成：先用大模型定布局，再生成带标签的点云。
- 3D高斯溅射结合概念引导，保持各物体身份清晰。
- 支持复杂场景如属性变化或物体互动，首例3D多概念定制。
尽管单概念3D定制已有研究，但多概念定制仍基本未被探索。为此，我们提出MultiDreamer3D，通过分而治之的方式生成连贯的多概念3D内容。首先，利用基于大语言模型的布局控制器生成3D边界框；其次，选择性点云生成器为每个概念创建粗略点云，并将其置入对应边界框中，初始化为带概念标签的3D高斯溅射（3D Gaussian Splatting），从而在2D投影中精确识别概念归属；最后，通过概念感知的区间评分匹配方法，结合概念感知扩散模型对3D高斯进行优化。实验表明，MultiDreamer3D不仅确保物体存在且保留各概念的独立身份，还能有效处理属性变化或物体交互等复杂情况。据我们所知，这是首个解决3D多概念定制问题的工作。
原文摘要 · Abstract (English)
While single-concept customization has been studied in 3D, multi-concept customization remains largely unexplored. To address this, we propose MultiDreamer3D that can generate coherent multi-concept 3D content in a divide-and-conquer manner. First, we generate 3D bounding boxes using an LLM-based layout controller. Next, a selective point cloud generator creates coarse point clouds for each concept. These point clouds are placed in the 3D bounding boxes and initialized into 3D Gaussian Splatting with concept labels, enabling precise identification of concept attributions in 2D projections. Finally, we refine 3D Gaussians via concept-aware interval score matching, guided by concept-aware diffusion. Our experimental results show that MultiDreamer3D not only ensures object presence and preserves the distinct identities of each concept but also successfully handles complex cases such as property change or interaction. To the best of our knowledge, we are the first to address the multi-concept customization in 3D.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。