提出新框架,用交叉验证评估合成数据质量。
Synthetic Dataset Evaluation Based on Generalized Cross Validation
- 结合广义交叉验证与领域迁移,构建可比评估体系。
- 在Virtual KITTI上验证,合成数据保真度显著提升。
- 适合研究合成数据生成与优化的学者使用。
随着合成数据生成技术的快速发展,评估合成数据质量已成为关键研究课题。稳健的评估不仅能推动数据生成方法的创新,还能指导研究人员优化合成数据的利用。然而,当前合成数据评估研究仍显不足,缺乏通用标准框架。为此,本文提出一种融合广义交叉验证实验与领域迁移学习原理的新评估框架,实现合成数据质量的可泛化、可比较评估。该框架在合成数据集和多个真实世界基准(如KITTI、BDD100K)上训练任务特定模型(如YOLOv5s),构建跨性能矩阵;经归一化后生成广义交叉验证(GCV)矩阵,量化领域迁移能力。框架引入两个核心指标:其一衡量合成数据与真实数据间的相似性以评估模拟质量;其二通过分析合成数据在多种真实场景中的多样性和覆盖度,评估迁移质量。在Virtual KITTI上的实验验证了所提框架与指标的有效性。该可扩展、可量化的评估方案克服了传统方法局限,为人工智能研究中合成数据优化提供了原则性路径。
原文摘要 · Abstract (English)
With the rapid advancement of synthetic dataset generation techniques, evaluating the quality of synthetic data has become a critical research focus. Robust evaluation not only drives innovations in data generation methods but also guides researchers in optimizing the utilization of these synthetic resources. However, current evaluation studies for synthetic datasets remain limited, lacking a universally accepted standard framework. To address this, this paper proposes a novel evaluation framework integrating generalized cross-validation experiments and domain transfer learning principles, enabling generalizable and comparable assessments of synthetic dataset quality. The framework involves training task-specific models (e.g., YOLOv5s) on both synthetic datasets and multiple real-world benchmarks (e.g., KITTI, BDD100K), forming a cross-performance matrix. Following normalization, a Generalized Cross-Validation (GCV) Matrix is constructed to quantify domain transferability. The framework introduces two key metrics. One measures the simulation quality by quantifying the similarity between synthetic data and real-world datasets, while another evaluates the transfer quality by assessing the diversity and coverage of synthetic data across various real-world scenarios. Experimental validation on Virtual KITTI demonstrates the effectiveness of our proposed framework and metrics in assessing synthetic data fidelity. This scalable and quantifiable evaluation solution overcomes traditional limitations, providing a principled approach to guide synthetic dataset optimization in artificial intelligence research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。