用规划代替生成,让3D场景编辑更精准可控
Edit-As-Act: Goal-Regressive Planning for Open-Vocabulary 3D Indoor Scene Editing
- 将编辑任务转为3D空间中的目标逆向规划,用符号化动作语言精确控制变化
- 在E2A-Bench上所有类型编辑任务均显著优于现有方法,保持语义一致与物理合理
- 适合需要高精度、可解释性3D场景修改的研究者和应用开发者
从自然语言编辑3D室内场景虽概念简单但技术挑战大。现有开放词汇系统常大面积重生成或依赖图像空间操作,破坏空间结构,导致意外全局变化或物理不一致。这源于将编辑视为生成任务。本文提出新视角:用户指令定义目标世界状态,编辑应是使该状态成立的最小动作序列,同时保留其余部分。基于此,我们提出Edit-As-Act框架,将开放词汇场景编辑建模为3D空间中的目标逆向规划。给定源场景与自由格式指令,模型预测符号化目标谓词,并在自研的EditLang(受PDDL启发)动作语言中规划,显式编码支持、接触、碰撞等几何关系。语言驱动规划器生成动作,验证器确保目标导向性、单调性与物理可行性,产出可解释且物理一致的变换。通过分离推理与低层生成,系统实现指令保真度、语义一致性与物理合理性三大指标的统一。在包含9个室内环境的63项任务的E2A-Bench上,本方法在所有编辑类型和场景类别中均显著优于先前方法。
原文摘要 · Abstract (English)
Editing a 3D indoor scene from natural language is conceptually straightforward but technically challenging. Existing open-vocabulary systems often regenerate large portions of a scene or rely on image-space edits that disrupt spatial structure, resulting in unintended global changes or physically inconsistent layouts. These limitations stem from treating editing primarily as a generative task. We take a different view. A user instruction defines a desired world state, and editing should be the minimal sequence of actions that makes this state true while preserving everything else. This perspective motivates Edit-As-Act, a framework that performs open-vocabulary scene editing as goal-regressive planning in 3D space. Given a source scene and free-form instruction, Edit-As-Act predicts symbolic goal predicates and plans in EditLang, a PDDL-inspired action language that we design with explicit preconditions and effects encoding support, contact, collision, and other geometric relations. A language-driven planner proposes actions, and a validator enforces goal-directedness, monotonicity, and physical feasibility, producing interpretable and physically coherent transformations. By separating reasoning from low-level generation, Edit-As-Act achieves instruction fidelity, semantic consistency, and physical plausibility - three criteria that existing paradigms cannot satisfy together. On E2A-Bench, our benchmark of 63 editing tasks across 9 indoor environments, Edit-As-Act significantly outperforms prior approaches across all edit types and scene categories.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。