arXiv:2412.00091cs.CVcs.AI2024-12被引 12

用图结构动态生成可调控的3D场景,无需重训

Graph Canvas for Controllable 3D Scene Generation

论文配图:Graph Canvas for Controllable 3D Scene Generation
图 1 · 摘自论文原文
  • 用分层图结构表示3D物体与关系,支持实时调整
  • 通过上下文学习实现动态适应,无需重新训练
  • 支持4D时序变化,适合交互式场景设计

空间智能是与物理世界互动的AI系统的基础,尤其在3D场景生成与空间理解中至关重要。当前3D场景生成方法多依赖预定义数据集,难以动态适应空间关系的变化。本文提出GraphCanvas3D,一个可编程、可扩展、可自适应的可控3D场景生成框架。该框架利用上下文学习,实现无需重训的动态适应,支持灵活定制的场景构建。其采用分层图驱动的场景描述,将空间元素表示为图节点,并建立3D环境中的对象间连贯关系。与传统方法相比,GraphCanvas3D摆脱了对预设输入掩码或重训的依赖,支持即时对象操作与场景调整。此外,该框架还支持4D场景生成,引入时间维度以建模随时间变化的动态过程。实验结果与用户研究证明,GraphCanvas3D显著提升了场景生成的可用性、灵活性与适应性。代码与模型已开源:https://github.com/ILGLJ/Graph-Canvas。

原文摘要 · Abstract (English)

Spatial intelligence is foundational to AI systems that interact with the physical world, particularly in 3D scene generation and spatial comprehension. Current methodologies for 3D scene generation often rely heavily on predefined datasets, and struggle to adapt dynamically to changing spatial relationships. In this paper, we introduce GraphCanvas3D, a programmable, extensible, and adaptable framework for controllable 3D scene generation. Leveraging in-context learning, GraphCanvas3D enables dynamic adaptability without the need for retraining, supporting flexible and customizable scene creation. Our framework employs hierarchical, graph-driven scene descriptions, representing spatial elements as graph nodes and establishing coherent relationships among objects in 3D environments. Unlike conventional approaches, which are constrained in adaptability and often require predefined input masks or retraining for modifications, GraphCanvas3D allows for seamless object manipulation and scene adjustments on the fly. Additionally, GraphCanvas3D supports 4D scene generation, incorporating temporal dynamics to model changes over time. Experimental results and user studies demonstrate that GraphCanvas3D enhances usability, flexibility, and adaptability for scene generation. Our code and models are available on the project website: https://github.com/ILGLJ/Graph-Canvas.

3D生成图神经网络可控生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。