用大规模数据和稀疏扩散模型，实现千平方公里级地球3D生成。
EarthCrafter: Scalable 3D Earth Generation via Dual-Sparse Latent Diffusion
- 分步生成地形与纹理，通过稀疏潜空间降低计算开销。
- 基于50,000个场景、4500万帧的航空数据集训练，支持超大规模生成。
- 适合地理建模、城市规划等需真实地形的科研与工程应用。
尽管近期3D生成技术取得显著进展，但将其扩展至地理尺度（如建模数千平方公里的地球表面）仍是未解难题。本文提出双创新方案：首先构建迄今最大的3D航空数据集Aerial-Earth3D，包含50,000个经筛选的场景（每个600m×600m），覆盖美国本土，含4500万张多视角谷歌地球图像，每场景提供带位姿标注的多视图图像、深度图、法向量、语义分割及相机位姿，并通过严格质量控制保障地形多样性。在此基础上，提出EarthCrafter框架，采用稀疏解耦潜空间扩散模型进行大规模3D地球生成。其架构分离结构与纹理生成：1）双稀疏3D-VAE将高分辨率几何体素与纹理2D高斯溅射（2DGS）压缩至紧凑潜空间，显著缓解地理尺度下的计算负担，同时保留关键信息；2）设计条件感知流匹配模型，基于混合输入（语义、图像或无）独立建模潜空间中的几何与纹理特征。大量实验表明，EarthCrafter在极端大规模生成中表现优异。该框架还可支持语义引导的城市布局生成与无条件地形合成，依托Aerial-Earth3D丰富的数据先验保持地理合理性。项目主页见https://whiteinblue.github.io/earthcrafter/
原文摘要 · Abstract (English)
Despite the remarkable developments achieved by recent 3D generation works, scaling these methods to geographic extents, such as modeling thousands of square kilometers of Earth's surface, remains an open challenge. We address this through a dual innovation in data infrastructure and model architecture. First, we introduce Aerial-Earth3D, the largest 3D aerial dataset to date, consisting of 50k curated scenes (each measuring 600m x 600m) captured across the U.S. mainland, comprising 45M multi-view Google Earth frames. Each scene provides pose-annotated multi-view images, depth maps, normals, semantic segmentation, and camera poses, with explicit quality control to ensure terrain diversity. Building on this foundation, we propose EarthCrafter, a tailored framework for large-scale 3D Earth generation via sparse-decoupled latent diffusion. Our architecture separates structural and textural generation: 1) Dual sparse 3D-VAEs compress high-resolution geometric voxels and textural 2D Gaussian Splats (2DGS) into compact latent spaces, largely alleviating the costly computation suffering from vast geographic scales while preserving critical information. 2) We propose condition-aware flow matching models trained on mixed inputs (semantics, images, or neither) to flexibly model latent geometry and texture features independently. Extensive experiments demonstrate that EarthCrafter performs substantially better in extremely large-scale generation. The framework further supports versatile applications, from semantic-guided urban layout generation to unconditional terrain synthesis, while maintaining geographic plausibility through our rich data priors from Aerial-Earth3D. Our project page is available at https://whiteinblue.github.io/earthcrafter/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。