自动规划视点生成户外3D场景,支持物体级控制。
SceneFrom3D: Geometry-Conditioned Outdoor 3D Scene Generation via View Scheduling with Object-Level Control

- 构建带方向的生成图自动规划视点与插值顺序
- 在真实户外场景上生成高质量3D结构,几何一致性高
- 支持物体外观与位置的精细控制,适合场景设计应用
几何条件化3D场景生成可基于用户提供的几何结构创建3D环境,实现对场景结构和物体布局的直接控制。现有方法通常采用三阶段流程:先定义视点调度,再沿预定视点生成多视角图像,最后从生成图像重建3D表示。然而,在户外场景中,由于几何体大、无序且无限延伸,难以获取能充分覆盖且支持稳定生成的视点,导致视点调度成为主要瓶颈。为此,我们提出SceneFrom3D框架,能够从户外输入几何中自动调度视点。该框架构建一个有向生成图,节点代表锚点视点,边代表插值轨迹,明确需合成的视点、需插值的视点对以及生成顺序。除自动视点调度外,SceneFrom3D还通过物体级条件增强可控性:为每个物体分配身份图像以引导外观,并设置几何贴合参数以实现区域级输入几何控制。实验表明,SceneFrom3D在几何条件化户外3D场景生成上达到当前最优性能,生成的场景质量高,且物体外观与几何贴合度可控。
原文摘要 · Abstract (English)
Geometry-conditioned 3D scene generation enables the creation of 3D environments from user-provided geometry, offering direct control over scene structure and object layout. To generate such 3D scenes, current methods commonly adopt a three-stage design that first defines a view schedule, then synthesizes multi-view observations along the scheduled views, and finally reconstructs a 3D representation from the generated images. However, defining the view schedule becomes a major bottleneck for outdoor scenes, where large, unstructured, and unbounded geometry makes it difficult to obtain views that provide sufficient coverage while supporting stable generation. To address this bottleneck, we present SceneFrom3D, a framework that automatically schedules views from outdoor input geometries. SceneFrom3D constructs a directed generation graph whose nodes represent anchor views and whose edges represent interpolation trajectories, defining which views to synthesize, which view pairs to interpolate, and in which order generation should proceed. Beyond automatic view scheduling, SceneFrom3D further improves controllability through object-level conditioning, assigning each object an identity image for appearance guidance and a geometry-adherence parameter for region-wise control over the input geometry. Experiments demonstrate that SceneFrom3D achieves state-of-the-art geometry-conditioned outdoor 3D scene generation, producing high-quality scenes with controllable object appearance and geometry adherence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。