用大模型生成稀有交通参与者数据，提升自动驾驶感知能力
LTDA-Drive: LLMs-guided Generative Models based Long-tail Data Augmentation for Autonomous Driving
- 用大模型指导扩散模型替换场景中的常见物体为罕见类
- 在KITTI上使稀有类别检测性能提升34.75%
- 适合关注长尾问题与数据增强的自动驾驶研究者
3D感知对提升自动驾驶的安全性与性能至关重要。然而，基于真实世界数据集训练的现有模型因数据长尾分布，往往在罕见且高风险的脆弱类别（如行人、骑行者）上表现不佳。现有重加权与重采样方法难以解决尾部类别样本稀缺与多样性不足的问题。为此，我们提出LTDA-Drive，一种由大语言模型引导的生成式数据增强框架，用于合成多样且高质量的长尾样本。该框架通过三阶段流程实现：(1) 文本引导的扩散模型移除驾驶场景中的头类物体；(2) 生成模型插入尾类实例；(3) 大语言模型代理过滤低质量合成图像。在KITTI数据集上的实验表明，LTDA-Drive显著提升了尾类检测性能，相较于基线方法，稀有类别检测准确率提升34.75%。结果进一步验证了该方法在生成高质量、多样化数据以应对长尾挑战方面的有效性。
原文摘要 · Abstract (English)
3D perception plays an essential role for improving the safety and performance of autonomous driving. Yet, existing models trained on real-world datasets, which naturally exhibit long-tail distributions, tend to underperform on rare and safety-critical, vulnerable classes, such as pedestrians and cyclists. Existing studies on reweighting and resampling techniques struggle with the scarcity and limited diversity within tail classes. To address these limitations, we introduce LTDA-Drive, a novel LLM-guided data augmentation framework designed to synthesize diverse, high-quality long-tail samples. LTDA-Drive replaces head-class objects in driving scenes with tail-class objects through a three-stage process: (1) text-guided diffusion models remove head-class objects, (2) generative models insert instances of the tail classes, and (3) an LLM agent filters out low-quality synthesized images. Experiments conducted on the KITTI dataset show that LTDA-Drive significantly improves tail-class detection, achieving 34.75\% improvement for rare classes over counterpart methods. These results further highlight the effectiveness of LTDA-Drive in tackling long-tail challenges by generating high-quality and diverse data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。