生成无限延伸的4D城市,区分动态车辆与静态建筑,效果逼真。
Compositional Generative Model of Unbounded 4D Cities
- 分模块生成:动态交通与静态布局分离,用神经场组合构建场景。
- 支持实例编辑、城市风格化与仿真,生成效果达当前最优。
- 基于真实地图数据,适配城市设计、自动驾驶等应用场景。
3D场景生成近年来发展迅速,但生成4D城市更具挑战性,因城市中包含结构复杂、视觉多样的建筑与车辆,且人类对城市畸变更敏感。为此,我们提出CityDreamer4D,一种专为生成无限延伸4D城市设计的组合式生成模型。核心思想是将动态物体(如车辆)与静态场景(如建筑和道路)分离,并采用不同类型的神经场表示建筑、车辆与背景元素。我们设计了交通场景生成器与无界布局生成器,通过紧凑的鸟瞰图(BEV)表示生成动态交通与静态城市布局。4D城市中的物体通过面向背景的神经场与面向实例的神经场组合生成。针对背景与实例特性,分别使用定制的生成哈希网格与周期性位置编码作为场景参数化方式。此外,我们提供了涵盖OSM、GoogleEarth和CityTopia的综合数据集:OSM提供多样真实城市布局,GoogleEarth与CityTopia则提供大规模高质量城市图像及带3D实例标注的图像。得益于其组合式设计,CityDreamer4D可支持实例编辑、城市风格化与城市仿真等多种下游应用,生成效果达到当前最优水平。
原文摘要 · Abstract (English)
3D scene generation has garnered growing attention in recent years and has made significant progress. Generating 4D cities is more challenging than 3D scenes due to the presence of structurally complex, visually diverse objects like buildings and vehicles, and heightened human sensitivity to distortions in urban environments. To tackle these issues, we propose CityDreamer4D, a compositional generative model specifically tailored for generating unbounded 4D cities. Our main insights are 1) 4D city generation should separate dynamic objects (e.g., vehicles) from static scenes (e.g., buildings and roads), and 2) all objects in the 4D scene should be composed of different types of neural fields for buildings, vehicles, and background stuff. Specifically, we propose Traffic Scenario Generator and Unbounded Layout Generator to produce dynamic traffic scenarios and static city layouts using a highly compact BEV representation. Objects in 4D cities are generated by combining stuff-oriented and instance-oriented neural fields for background stuff, buildings, and vehicles. To suit the distinct characteristics of background stuff and instances, the neural fields employ customized generative hash grids and periodic positional embeddings as scene parameterizations. Furthermore, we offer a comprehensive suite of datasets for city generation, including OSM, GoogleEarth, and CityTopia. The OSM dataset provides a variety of real-world city layouts, while the Google Earth and CityTopia datasets deliver large-scale, high-quality city imagery complete with 3D instance annotations. Leveraging its compositional design, CityDreamer4D supports a range of downstream applications, such as instance editing, city stylization, and urban simulation, while delivering state-of-the-art performance in generating realistic 4D cities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。