通过学习物体间局部关系,实现更真实的3D室内场景生成。
Pair2Scene: Learning Local Object Relations for Procedural Scene Generation

- 基于局部依赖关系建模物体间的支撑与功能联系。
- 在3D-Pairs数据集上训练,生成场景超越训练分布且符合物理逻辑。
- 适合需要高真实感场景生成的虚拟现实与游戏开发人员。
由于数据稀缺和复杂空间关系建模困难,生成高保真3D室内场景仍具挑战。现有方法常难以扩展至密集场景,或依赖缺乏精确空间推理能力的LLM/VLM。本文观察到物体摆放主要依赖局部依赖而非冗余全局分布,提出Pair2Scene框架,融合学习到的局部规则、场景层次结构与物理算法。规则捕捉两类物体间关系:遵循物理层级的支撑关系,以及体现语义关联的功能关系。通过网络估计依赖物体相对于锚定物体的位置分布。构建3D-Pairs数据集用于训练模型。推理时,框架在层次结构中递归应用模型,结合碰撞感知拒绝采样,将局部规则整合为连贯全局布局。大量实验表明,该框架在生成超出训练数据范围的复杂环境方面优于现有方法,同时保持物理与语义合理性。
原文摘要 · Abstract (English)
Generating high-fidelity 3D indoor scenes remains a significant challenge due to data scarcity and the complexity of modeling intricate spatial relations. Current methods often struggle to scale beyond training distribution to dense scenes or rely on LLMs/VLMs that lack the ability for precise spatial reasoning. Building on top of the observation that object placement relies mainly on local dependencies instead of information-redundant global distributions, in this paper, we propose Pair2Scene, a novel procedural generation framework that integrates learned local rules with scene hierarchies and physics-based algorithms. These rules mainly capture two types of inter-object relations, namely support relations that follow physical hierarchies, and functional relations that reflect semantic links. We model these rules through a network, which estimates spatial position distributions of dependent objects conditioned on position and geometry of the anchor ones. Accordingly, we curate a dataset 3D-Pairs from existing scene data to train the model. During inference, our framework can generate scenes by recursively applying our model within a hierarchical structure, leveraging collision-aware rejection sampling to align local rules into coherent global layouts. Extensive experiments demonstrate that our framework outperforms existing methods in generating complex environments that go beyond training data while maintaining physical and semantic plausibility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。