用分层语言让大模型更准地生成和修改3D室内场景。
HDSL: A Hierarchical Domain-Specific Language for Structured 3D Indoor Scene Generation and Localized Editing with LLM Agents

- 设计分层语言HDSL,用树形结构表达房间与物体位置关系。
- 生成时用多模态检索定位物体,编辑时局部修改不破坏整体。
- 编辑效率提升6倍以上,且能保持未改部分不变。
基于文本的室内场景生成与编辑需要语言模型可生成且可修改的中间表示。现有基于大模型的系统常依赖场景图或全局约束列表,虽紧凑但难以描述局部几何,导致指令式编辑难以精准定位。本文将该问题建模为结构化程序生成与局部程序修复,提出一种类XML/CSS的领域专用语言HDSL,用于结构化3D室内场景。HDSL以树形结构表示房间、区域、物体和支持面,并包含局部坐标,使复杂场景易于递归规划与检索编辑。我们的流水线利用大模型代理生成受限验证的HDSL子树,通过多模态资产检索定位非虚拟节点,并使用力导向布局优化修复边界与碰撞错误。对于编辑,采用分层检索增强生成(HRAG)定位相关子树,仅重写局部上下文后通过确定性三路合并回原结构。在复现基准测试中,HDSL在对象覆盖率、文本-场景对齐度和生成时间上优于全文本到场景基线,同时在几何指标上与近期仅布局还原方法相当;在编辑方面,HRAG减少5.22倍的令牌使用量与6.19倍的运行时间,所有八组配对编辑均生成有效DSL,并更好保留无关场景对象。
原文摘要 · Abstract (English)
Text-driven indoor scene generation and editing require an intermediate representation that language models can both produce and revise. Existing LLM-based systems often rely on scene graphs or global constraint lists, which are compact but underspecify local geometry and make instruction-based edits difficult to localize. We frame this problem as structured program generation and local program repair, and propose Hierarchical Descriptive Scene Language (HDSL), an XML/CSS-style domain-specific language for structured 3D indoor scenes. HDSL represents rooms, regions, objects, and support surfaces as a tree with local coordinates, making complex scenes easier to plan recursively and easier to retrieve for editing. Our pipeline uses LLM agents to generate HDSL subtrees with bounded verification, grounds non-virtual nodes through multimodal asset retrieval, and applies force-directed layout optimization to repair boundary and collision errors. For editing, Hierarchical Retrieval-Augmented Generation retrieves the relevant subtree, asks the LLM to rewrite only that local context, and merges the result back through a deterministic three-way merge. In our reproduced benchmark, HDSL improves average object coverage, text-scene alignment, and generation time over full text-to-scene baselines while remaining competitive with recent layout-only reproductions on geometry metrics; for editing, HRAG reduces token use by $5.22\times$ and runtime by $6.19\times$, produces valid DSL for all eight paired edits, and better preserves unrelated scene objects.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。