提出新方法评估合成少数样本有效性,发现多数生成方法实际无效。
Synthetic minority data is redundant or invalid: a data-dependent validity theory and a de-biased test

- 用保留真实数据检验合成样本归属概率,避免自检偏差
- 96%-99%情况下传统检验严重低估无效性,新方法更准确
- 证明合成数据有效性取决于数据本身,非生成方法能力
过去二十年,类别不平衡学习的标准解决方案是生成合成少数类样本,其有效性验证依赖于将合成点与生成它们的数据对比——这种检验必然通过。本文提出去偏验证方法:有效性变为可估计的总体指标,即合成点真正属于少数类的概率,通过在保留的真实数据上评分实现。当有保留真实标签时,传统检验在96-99%的方法-不平衡比组合中严重低估真实无效性,而新估计器能精准跟踪。我们证明有效性是数据属性而非方法属性:类别重叠设定了无法回避的无效下限,当类别可分时过采样冗余,重叠时则无效。在91种方法、3种分类器及医疗与金融数据集上的实验显示,即使某些生成器通过了传统检验,也均未同时满足有效性和信息增益双标准:相比最优平凡基线的提升极微弱(中位F1低于0.01,仅达决策阈值),且多数损害校准性能。我们发布可安装的审计工具,将举证责任反转:合成少数数据必须在具体数据上证明自身有效且带来真实信息增益。
原文摘要 · Abstract (English)
For two decades, the standard remedy for class-imbalanced learning has been to fabricate synthetic minority examples, and the standard evidence of their validity has been a check that cannot fail: synthetic points are scored against the very data that generated them. We de-bias the check. Validity becomes a population quantity -- the probability that a synthetic point truly belongs to the minority class -- with a consistent estimator that scores synthetic points against withheld real data. Where held-out ground truth is available, the classical test underestimates true invalidity in 96-99% of method-by-imbalance-ratio cells, while the de-biased estimator tracks it closely. We prove validity is a property of the data, not the method: class overlap sets an invalidity floor no faithful generator escapes, making oversampling redundant where classes separate and invalid where they overlap. Across 91 methods, three classifiers, and datasets spanning medicine and finance -- including a generator engineered to pass the classical check -- none clears both bars: gains over the best trivial baseline are noise-thin (median below 0.01 F1, a decision threshold's reach), and most damage calibration. We release the audit as a pip-installable test and flip the burden of proof: synthetic minority data must now demonstrate, on the data at hand, both validity and information gain.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。