用草图生成真实城市3D场景，提升空间一致性与细节还原
CymbaDiff: Structured Spatial Diffusion for Sketch-based 3D Semantic Urban Scene Generation
- 基于草图和卫星图构建结构化扩散模型，显式建模柱状连续性与垂直层级
- 在SketchSem3D上实现更高语义一致性与跨数据集泛化能力
- 适合自动驾驶、城市仿真等需要高保真3D场景的研究者
室外3D语义场景生成为城市模拟和自动驾驶等应用提供逼真且语义丰富的环境。然而，该方向进展受限于缺乏公开可用且标注良好的数据集。我们提出SketchSem3D，首个从抽象手绘草图和伪标注卫星图像生成3D户外语义场景的大规模基准数据集。SketchSem3D包含两个子集：基于草图的SemanticKITTI和基于草图的KITTI-360（包含激光雷达体素及其对应草图与标注卫星图像），支持标准化、严格且多样的评估。我们还提出Cylinder Mamba Diffusion（CymbaDiff），显著提升室外3D场景生成的空间一致性。CymbaDiff引入结构化空间排序，显式捕捉柱状连续性和垂直层次，并保留生成场景中的物理邻近关系与全局上下文。在SketchSem3D上的大量实验表明，CymbaDiff在语义一致性、空间真实性及跨数据集泛化方面均表现卓越。代码与数据集将开源。
原文摘要 · Abstract (English)
Outdoor 3D semantic scene generation produces realistic and semantically rich environments for applications such as urban simulation and autonomous driving. However, advances in this direction are constrained by the absence of publicly available, well-annotated datasets. We introduce SketchSem3D, the first large-scale benchmark for generating 3D outdoor semantic scenes from abstract freehand sketches and pseudo-labeled annotations of satellite images. SketchSem3D includes two subsets, Sketch-based SemanticKITTI and Sketch-based KITTI-360 (containing LiDAR voxels along with their corresponding sketches and annotated satellite images), to enable standardized, rigorous, and diverse evaluations. We also propose Cylinder Mamba Diffusion (CymbaDiff) that significantly enhances spatial coherence in outdoor 3D scene generation. CymbaDiff imposes structured spatial ordering, explicitly captures cylindrical continuity and vertical hierarchy, and preserves both physical neighborhood relationships and global context within the generated scenes. Extensive experiments on SketchSem3D demonstrate that CymbaDiff achieves superior semantic consistency, spatial realism, and cross-dataset generalization. The code and dataset will be available at https://github.com/Lillian-research-hub/CymbaDiff
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。