arXiv:2506.02459cs.CV2025-06被引 3

用自然语言生成和编辑3D室内场景,支持物体增删换且保持空间合理。

ReSpace: Text-Driven Autoregressive 3D Indoor Scene Synthesis and Editing

  • 基于自回归文本驱动,显式建模房间边界与物体语义。
  • 在物体添加任务上超越现有方法,合成场景人眼质量更优。
  • 适合需要自然语言交互的3D场景生成与编辑应用。

场景生成与编辑已成为计算机图形学中的前沿方向。现有3D室内场景生成方法或使用单一类别编码简化语义(如‘椅子’或‘桌子’),或需掩码扩散模型进行编辑,忽略房间边界,或依赖平面图渲染而无法捕捉复杂布局。基于大语言模型的方法虽能通过自然语言实现丰富语义,但缺乏编辑功能,仅限矩形布局,或依赖隐式世界模型的弱空间推理。我们提出ReSpace,一个自回归、文本驱动的3D室内场景生成与编辑框架。该方法采用紧凑的结构化场景表示,显式建模房间边界,实现无资产依赖部署,并将场景操作转化为下一词预测任务,支持通过自然语言实现物体的添加、删除与替换。我们采用监督微调结合偏好对齐训练专用语言模型,以兼顾用户指令、空间几何、物体语义与整体构图。此外,引入基于体素的评估指标,捕捉超出3D包围盒的精细几何违规。实验表明,在物体添加任务上优于现有最佳方法,并在全场景生成应用中获得更优的人类感知质量,尽管未在该任务上进行训练。

原文摘要 · Abstract (English)

Scene synthesis and editing has emerged as a promising direction in computer graphics. Current trained approaches for 3D indoor scene generation either oversimplify object semantics through one-hot class encodings (e.g., 'chair' or 'table'), require masked diffusion for editing, ignore room boundaries, or rely on floor plan renderings that fail to capture complex layouts. LLM-based methods enable richer semantics via natural language, but lack editing functionality, are limited to rectangular layouts, or rely on weak spatial reasoning from implicit world models. We introduce ReSpace, a generative framework for autoregressive text-driven 3D indoor scene synthesis and editing. Our approach features a compact structured scene representation with explicit room boundaries that enables asset-agnostic deployment and frames scene manipulation as a next-token prediction task, supporting object addition, removal, and swapping via natural language. We employ supervised fine-tuning with a preference alignment stage to train a specialized language model for object addition that accounts for user instructions, spatial geometry, object semantics, and scene-level composition. We further introduce a voxelization-based evaluation metric capturing fine-grained geometric violations beyond 3D bounding boxes. Experiments surpass state-of-the-art on object addition and achieve superior human-perceived quality on the application of full scene synthesis, despite not being trained on it.

3D生成文本生成场景编辑自回归

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。