arXiv:2608.27996cs.AI2026-08

用少量真实数据判断合成数据是否有效,避免训练浪费。

Should I Use This Synthetic Dataset for Training? How to Test with Minimal Real Data

论文配图:Should I Use This Synthetic Dataset for Training? How to Test with Minimal Real Data
图 1 · 摘自论文原文
  • 提出自适应符号翻转检验,动态调整测试计算量和真实数据用量。
  • 实验表明只需极少真实样本即可识别出有效的合成数据。
  • 适合在真实数据稀缺时评估合成数据价值,尤其适用于工程系统AI训练。

数字孪生(DTs)和学习型世界模型正被广泛用于生成合成数据,以补充工程系统中稀缺的真实数据。然而,由于不可避免的仿真到现实(sim-to-real)差距,合成数据增强可能无法提升模型在真实数据分布上的性能。本文解决这一决策问题:给定真实数据、候选合成数据及固定学习算法,判断在增广数据上训练是否能提升模型的总体性能,同时尽可能减少真实测试数据的使用。本文考虑两种方法:基于均值损失差的直接检验,以及基于对称性的符号翻转检验,后者以更强的零假设换取更快证据积累。为此引入自适应e过程符号翻转检验(aeSFT),一种双重自适应方法,可动态调整蒙特卡洛符号翻转轮数(影响计算成本)和真实测试数据消耗量。aeSFT实现任意时间有效的第一类错误控制,无需预先指定测试集大小。在合成分类任务、基于数字孪生的无线包调度任务和无线地图预测任务上的实验表明,aeSFT比基于均值的序列检验使用更少的真实测试样本,同时保持与固定样本符号翻转检验和配对t检验相当的检验功效,且假阳性率低于目标水平。

原文摘要 · Abstract (English)

Digital twins (DTs) and learned world models are increasingly used to generate synthetic data that augment the scarce real datasets available for training artificial intelligence (AI) models in engineering systems. Owing to the inevitable simulation-to-reality (sim-to-real) gap, however, augmentation may fail to improve the performance of the trained model on the real data distribution. This paper addresses the resulting decision problem: Given a real dataset, a candidate synthetic dataset, and a fixed learning algorithm, decide whether training on the augmented dataset improves the true, population-level performance, while consuming as few real test data points as possible. Two formulations are considered: a direct test on the mean loss difference between the two trained models, and a symmetry-based test on the paired loss difference, which trades a stronger null assumption for faster evidence accumulation. For the latter, we introduce the {adaptive e-process sign-flip test} (aeSFT), a doubly adaptive procedure that adapts both the number of Monte Carlo sign-flip rounds, and hence the computational cost, and the amount of real test data consumed. aeSFT yields anytime-valid Type-I error control, with no need to pre-specify the test-set size. Experiments on a synthetic-data classification task, a DT-aided wireless packet-scheduling task, and a radio-map prediction task show that aeSFT identifies useful synthetic data using substantially fewer real test samples than mean-based sequential testing, matches the power of fixed-sample sign-flip testing and the paired $t$-test, while keeping the false-positive rate below the target level.

合成数据测试方法数字孪生小样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。