用一张图或一段话生成完整无碰撞的3D房间,还能自由调整布局。
RoomCraft: Controllable and Complete 3D Indoor Scene Generation
- 通过结构化信息解析用户输入,生成有逻辑关系的家具布局。
- 采用启发式搜索与动态权重调整,避免家具重叠并保证布局完整。
- 支持图像、草图、文字多种输入,适合场景设计与虚拟装修应用。
从用户输入生成逼真的3D室内场景仍是计算机视觉与图形学中的难题,需兼顾几何一致性、空间关系和视觉真实感。现有神经生成方法因全局空间推理能力有限,常产生重复元素;而传统程序化方法虽可控制生成,但在多约束条件下易发生物体碰撞,导致家具被移除,影响布局完整性。为此,我们提出RoomCraft,一个分阶段的多模态生成框架,能将真实图像、草图或文本描述转化为连贯的3D室内场景。该框架首先从用户输入中提取高层语义信息,构建包含房间类型、家具种类及空间关系的结构化表示;随后建立空间关系网络,利用基于启发式的深度优先搜索(HDFS)算法生成优化的家具摆放序列,确保布局一致性。为应对复杂多约束场景,我们设计统一的约束表达方式,兼容形式化规范与自然语言输入,并通过全面的动作空间设计实现灵活调整。此外,提出冲突感知定位策略(CAPS),动态调整放置权重以最小化家具碰撞,保障布局完整性。大量实验表明,RoomCraft在多种输入模态下均显著优于现有方法,在生成真实、语义一致且视觉吸引人的房间布局方面表现优异。
原文摘要 · Abstract (English)
Generating realistic 3D indoor scenes from user inputs remains a challenging problem in computer vision and graphics, requiring careful balance of geometric consistency, spatial relationships, and visual realism. While neural generation methods often produce repetitive elements due to limited global spatial reasoning, procedural approaches can leverage constraints for controllable generation but struggle with multi-constraint scenarios. When constraints become numerous, object collisions frequently occur, forcing the removal of furniture items and compromising layout completeness. To address these limitations, we propose RoomCraft, a multi-stage pipeline that converts real images, sketches, or text descriptions into coherent 3D indoor scenes. Our approach combines a scene generation pipeline with a constraint-driven optimization framework. The pipeline first extracts high-level scene information from user inputs and organizes it into a structured format containing room type, furniture items, and spatial relations. It then constructs a spatial relationship network to represent furniture arrangements and generates an optimized placement sequence using a heuristic-based depth-first search (HDFS) algorithm to ensure layout coherence. To handle complex multi-constraint scenarios, we introduce a unified constraint representation that processes both formal specifications and natural language inputs, enabling flexible constraint-oriented adjustments through a comprehensive action space design. Additionally, we propose a Conflict-Aware Positioning Strategy (CAPS) that dynamically adjusts placement weights to minimize furniture collisions and ensure layout completeness. Extensive experiments demonstrate that RoomCraft significantly outperforms existing methods in generating realistic, semantically coherent, and visually appealing room layouts across diverse input modalities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。