用自然语言控制室内3D场景风格生成，提升设计自动化水平
Decorum: A Language-Based Approach For Style-Conditioned Synthesis of Indoor 3D Scenes
- 全程使用语言表示，实现从文本到场景的端到端控制
- 在3D-FRONT数据集上，文本条件生成与物体检索效果优于现有方法
- 适合需要快速原型设计的建筑师与家居设计师
3D室内场景生成对数字与现实环境设计至关重要。为实现自动化，场景生成模型不仅需生成合理布局，还需考虑视觉特征与风格偏好。现有方法对这些属性的控制能力有限，仅支持简单的物体描述或空间关系输入。本文提出Decorum，通过在每个阶段采用语言表征，使用户能以自然语言控制生成过程，利用大语言模型（LLM）建模语言到语言的映射。此外，我们提出一种基于多模态大模型的新物体检索方法，用于场景家具选择。在基准数据集3D-FRONT上的评估显示，该方法在文本条件场景合成与物体检索任务中均优于现有工作。
原文摘要 · Abstract (English)
3D indoor scene generation is an important problem for the design of digital and real-world environments. To automate this process, a scene generation model should be able to not only generate plausible scene layouts, but also take into consideration visual features and style preferences. Existing methods for this task exhibit very limited control over these attributes, only allowing text inputs in the form of simple object-level descriptions or pairwise spatial relationships. Our proposed method Decorum enables users to control the scene generation process with natural language by adopting language-based representations at each stage. This enables us to harness recent advancements in Large Language Models (LLMs) to model language-to-language mappings. In addition, we show that using a text-based representation allows us to select furniture for our scenes using a novel object retrieval method based on multimodal LLMs. Evaluations on the benchmark 3D-FRONT dataset show that our methods achieve improvements over existing work in text-conditioned scene synthesis and object retrieval.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。