arXiv:2608.03064cs.CV2026-08

用图结构全局优化3D室内布局生成,让场景更合理更一致。

Global Graph-Validated Optimization for VLM-based 3D Indoor Scene Generation

论文配图:Global Graph-Validated Optimization for VLM-based 3D Indoor Scene Generation
图 1 · 摘自论文原文
  • 构建图结构中间表示,分离语义与物理约束
  • 结合进化搜索与梯度优化,提升全局一致性
  • 适合需要高物理合理性与语义一致性的场景生成

我们研究开放词汇的3D室内布局生成,即从无标注3D资产中,通过自由形式的语言指令合成多样且物理合理的场景。现有方法依赖大语言模型(LLMs)和视觉-语言模型(VLMs)生成结构化场景,但多数仅使用局部成对约束和局部优化,难以适应高度非凸的全局布局空间,常导致局部合理但全局不一致或物理不可行的场景。为此,我们提出基于图的中间表示,将语义一致性与物理可行性解耦,并采用混合搜索与精炼策略。首先,全局语义验证(GSV)以结构化图表示场景,通过规则验证强制语义约束,消除矛盾配置,生成全局一致的语义骨架。其次,全局物理可行性搜索(GPFS)结合进化搜索进行全局探索与梯度优化进行局部精炼,降低对VLM初始化的依赖,增强在非凸、不连续可行空间中的鲁棒性。实验表明,该方法在开放词汇3D室内布局生成上达到当前最优性能,显著提升语义一致性和物理合理性。

原文摘要 · Abstract (English)

We study open-vocabulary 3D indoor layout generation, which synthesizes diverse and physically plausible scenes from unlabeled 3D assets using free-form language instructions. Recent methods leverage large language models (LLMs) and vision-language models (VLMs) to generate structured scenes from text. However, most model inter-asset relations implicitly or rely on local pairwise constraints and local optimization. These formulations are poorly aligned with the global, highly non-convex layout space, often yielding locally plausible yet globally inconsistent or physically infeasible scenes. We address this problem with a graph-based intermediate representation that separates semantic coherence from physical feasibility, together with a hybrid search-and-refinement strategy. First, Global Semantic Verification (GSV) represents scenes as structured graphs and enforces semantic constraints through rule-based verification. This explicit validation removes contradictory configurations and produces a globally consistent semantic scaffold. Second, Global Physical Feasibility Search (GPFS) combines evolutionary search for global exploration with gradient-based refinement for local exploitation. It reduces dependence on VLM-proposed initialization and improves robustness in non-convex and discontinuous feasible spaces. Together, GSV and GPFS move layout generation beyond local relational modeling and initialization-sensitive optimization toward globally consistent reasoning and search. Experiments show that our method achieves state-of-the-art performance in open-vocabulary 3D indoor layout generation, improving both semantic consistency and physical plausibility.

3D生成视觉语言模型布局优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。