基于去噪的3D室内布局生成,精准控制家具位置与大小。
DeBaRA: Denoising-Based 3D Room Arrangement Generation
- 采用轻量级得分模型,聚焦3D空间属性建模。
- 单个模型支持场景生成、补全与重排等多种任务。
- 可与LLM结合使用,适合交互式设计应用。
生成真实且多样的带家具3D室内场景布局,可赋能众多交互式应用,涉及广泛行业。物体间复杂交互、可用数据有限及空间约束要求,使3D场景合成与布置的生成建模极具挑战。现有方法通常采用自回归或现成扩散目标,同时预测所有属性但缺乏3D推理。本文提出DeBaRA,一种专为受限环境中精确、可控、灵活布置生成设计的得分模型。我们认为场景合成系统最关键的是准确确定各类物体在限定区域内的尺寸与位置。基于此,我们设计了一种以3D空间感知为核心的轻量级条件得分模型。实验证明,通过聚焦物体的空间属性,单一训练好的DeBaRA模型可在测试时用于场景生成、补全和重排等下游任务。此外,我们提出一种新颖的自评分评估方法,使其能最优地与外部大语言模型协同工作。我们在大量实验中验证了该方法,在多种场景下显著优于当前最先进方法。
原文摘要 · Abstract (English)
Generating realistic and diverse layouts of furnished indoor 3D scenes unlocks multiple interactive applications impacting a wide range of industries. The inherent complexity of object interactions, the limited amount of available data and the requirement to fulfill spatial constraints all make generative modeling for 3D scene synthesis and arrangement challenging. Current methods address these challenges autoregressively or by using off-the-shelf diffusion objectives by simultaneously predicting all attributes without 3D reasoning considerations. In this paper, we introduce DeBaRA, a score-based model specifically tailored for precise, controllable and flexible arrangement generation in a bounded environment. We argue that the most critical component of a scene synthesis system is to accurately establish the size and position of various objects within a restricted area. Based on this insight, we propose a lightweight conditional score-based model designed with 3D spatial awareness at its core. We demonstrate that by focusing on spatial attributes of objects, a single trained DeBaRA model can be leveraged at test time to perform several downstream applications such as scene synthesis, completion and re-arrangement. Further, we introduce a novel Self Score Evaluation procedure so it can be optimally employed alongside external LLM models. We evaluate our approach through extensive experiments and demonstrate significant improvement upon state-of-the-art approaches in a range of scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。