测试生成数据质量指标能否预测目标检测性能,发现效果依赖具体任务和训练方式。
Do Generative Metrics Predict YOLO Performance? An Evaluation Across Models, Augmentation Ratios, and Dataset Complexity
- 对比多种生成模型在不同数据集上的增强效果,用真实检测指标验证
- 在行人和盆栽检测中生成数据提升达30.6%相对精度,交通标志提升不明显
- 强调不能仅看生成指标原始相关性,需控制增强比例才能准确评估
合成图像越来越多用于目标检测训练集的数据增强,但训练前可靠评估合成数据集仍具挑战:标准全局生成指标(如FID)常无法预测下游检测的mAP。本文针对YOLOv11,在三个单类检测场景下进行受控评估——交通标志(稀疏/近饱和)、Cityscapes行人(密集/遮挡严重)、COCO盆栽(多实例/高变异性)。在合成比例从10%到150%的真实训练集范围内,对六种基于GAN、扩散模型及混合架构的生成器进行测试,分别从零开始训练与使用COCO预训练初始化,并在保留的真实测试集上评估[email protected]:0.95。每个配置下通过匹配大小的自助抽样协议计算预训练数据集指标,包括(i)Inception-v3与DINOv2嵌入中的全局特征空间指标,以及(ii)边界框统计量上的对象中心分布距离。结果显示,在更复杂的任务中,合成数据增强带来显著收益(行人+7.6%,盆栽+30.6%相对提升),而在交通标志或预训练微调下效果有限。为区分指标信号与增强数量影响,报告原始与校正后(残差化)的相关性,并经多重检验校正,表明指标与性能的关联性强烈依赖任务场景,多数表面相关性在控制增强水平后减弱。
原文摘要 · Abstract (English)
Synthetic images are increasingly used to augment object-detection training sets, but reliably evaluating a synthetic dataset before training remains difficult: standard global generative metrics (e.g., FID) often do not predict downstream detection mAP. We present a controlled evaluation of synthetic augmentation for YOLOv11 across three single-class detection regimes -- Traffic Signs (sparse/near-saturated), Cityscapes Pedestrian (dense/occlusion-heavy), and COCO PottedPlant (multi-instance/high-variability). We benchmark six GAN-, diffusion-, and hybrid-based generators over augmentation ratios from 10% to 150% of the real training split, and train YOLOv11 both from scratch and with COCO-pretrained initialization, evaluating on held-out real test splits ([email protected]:0.95). For each dataset-generator-augmentation configuration, we compute pre-training dataset metrics under a matched-size bootstrap protocol, including (i) global feature-space metrics in both Inception-v3 and DINOv2 embeddings and (ii) object-centric distribution distances over bounding-box statistics. Synthetic augmentation yields substantial gains in the more challenging regimes (up to +7.6% and +30.6% relative mAP in Pedestrian and PottedPlant, respectively) but is marginal in Traffic Signs and under pretrained fine-tuning. To separate metric signal from augmentation quantity, we report both raw and augmentation-controlled (residualized) correlations with multiple-testing correction, showing that metric-performance alignment is strongly regime-dependent and that many apparent raw associations weaken after controlling for augmentation level.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。