用扩散模型生成雨夜等稀有场景图像，提升自动驾驶分割与决策性能。
SynDiff-AD: Improving Semantic Segmentation and End-to-End Autonomous Driving with Synthetic Data from Latent Diffusion Models
- 基于控制网和语义引导的生成式数据增强方法
- 在Waymo和DeepDrive上分割精度提升最高达2.3%
- 适用于需要鲁棒性感知的自动驾驶系统研发
近年来，大规模数据集的积累显著提升了分割与自动驾驶模型性能。然而，现有数据集多集中于晴天白天等常见环境，导致雨夜等少样本条件下的表现下降。为此，本文提出SynDiff-AD，一种基于潜空间扩散模型的数据增强流程，利用控制网（ControlNet）在语义图引导下生成真实感图像，并设计了针对子群体的语义密集型提示方案。通过在数据集中引入合成图像，我们使Mask2Former和SegFormer在Waymo数据集上的分割性能分别提升1.2%和2.3%，在DeepDrive上分别提升1.4%和0.7%。此外，该方法还将端到端自动驾驶模型（如AIM-2D和AIM-BEV）在CARLA模拟器中跨多种环境的驾驶表现提升最高达20%，显著增强了模型鲁棒性。代码已开源。
原文摘要 · Abstract (English)
In recent years, significant progress has been made in collecting large-scale datasets to improve segmentation and autonomous driving models. These large-scale datasets are often dominated by common environmental conditions such as "Clear and Day" weather, leading to decreased performance in under-represented conditions like "Rainy and Night". To address this issue, we introduce SynDiff-AD, a novel data augmentation pipeline that leverages diffusion models (DMs) to generate realistic images for such subgroups. SynDiff-AD uses ControlNet-a DM that guides data generation conditioned on semantic maps-along with a novel prompting scheme that generates subgroup-specific, semantically dense prompts. By augmenting datasets with SynDiff-AD, we improve the performance of segmentation models like Mask2Former and SegFormer by up to 1.2% and 2.3% on the Waymo dataset, and up to 1.4% and 0.7% on the DeepDrive dataset, respectively. Additionally, we demonstrate that our SynDiff-AD pipeline enhances the driving performance of end-to-end autonomous driving models, like AIM-2D and AIM-BEV, by up to 20% across diverse environmental conditions in the CARLA autonomous driving simulator, providing a more robust model. We release our code and pipeline at https://github.com/UTAustin-SwarmLab/SynDiff-AD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。