用2D扩散模型实现可编辑的3D语义场景生成
EditSSC: Toward Editable Semantic Occupancy Scenes with Unconditional Diffusion Models
- 将3D语义占用格子转为多通道俯视图图像，复用2D扩散模型
- 无需重训练即可支持草图引导、修复和扩展生成
- 在SemanticKITTI上超越现有3D专用方法
3D语义场景生成对自动驾驶至关重要，但现有方法多依赖复杂的3D专用架构（如三平面编码器和适配的扩散网络），限制了简洁性和可编辑性。本文提出EditSSC，一种基于2D鸟瞰图（BEV）表示与现成潜在扩散模型的可编辑3D语义场景生成方法。我们将3D语义占用网格转化为多通道BEV图像，仅小幅修改Stable Diffusion中的量化自编码器与UNet。在量化后的潜在空间进行扩散，实现无需训练的编辑能力。通过代码本中的类别-代码对应关系，方法支持草图引导生成、图像修复和外推生成，无需任何重训练。在SemanticKITTI数据集上，EditSSC在无条件生成任务中优于现有3D专用基线，证明成熟2D架构可有效复用于3D场景生成与编辑。
原文摘要 · Abstract (English)
3D semantic scene generation is crucial for autonomous driving applications, yet most methods rely on complex 3D-specific architectures such as triplane encoders and adapted diffusion networks, limiting both their simplicity and their editing capabilities. We propose EditSSC, an editing-ready method for 3D semantic scene generation using 2D Bird's Eye View (BEV) representations and off-the-shelf latent diffusion network. Our approach reshapes 3D semantic occupancy grids into multi-channel BEV images and leverages the quantized autoencoder and UNet from Stable Diffusion with minimal modifications. We perform diffusion on the latents after quantization, which enables training-free editing capabilities. By exploiting class-to-code correspondences in the codebook, our method supports sketch-guided generation, inpainting, and outpainting without any retraining. On SemanticKITTI, EditSSC outperforms existing 3D-specific baselines on unconditional generation, demonstrating that well-established 2D architectures can be effectively repurposed for 3D scene generation and editing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。