无需模型结构信息，用扰动导数法评估黑箱模型的过拟合风险。
Perturbing the Derivative: Doubly Wild Refitting for Model-Free Evaluation of Opaque Machine Learning Predictors
- 通过随机扰动导数生成伪标签数据，双次重训练得两个野性预测器。
- 在固定设计下给出过剩风险的高效上界，不依赖函数类复杂度。
- 适合评估深度网络等复杂黑箱模型，突破传统理论限制。
我们研究在凸损失下经验风险最小化（ERM）的过剩风险评估问题。通过利用野性重拟合的思想，可在不依赖底层函数类全局结构的前提下，仅需黑箱训练算法和单个数据集，通过所谓“野性乐观”来上界控制过剩风险。具体方法是：通过精心设计的缩放因子对导数进行随机扰动，生成两组人工修改的伪结果；基于这些伪标签数据，对黑箱过程进行两次重拟合，得到两个野性预测器，并在固定设计设定下推导出高效的过剩风险上界。该方法无需事先了解底层函数类的复杂度，本质上是模型无关的，对现代复杂且不透明的深度神经网络与生成模型具有重要理论评估价值，传统学习理论因假设类过于复杂而难以适用。
原文摘要 · Abstract (English)
We study the problem of excess risk evaluation for empirical risk minimization (ERM) under convex losses. We show that by leveraging the idea of wild refitting, one can upper bound the excess risk through the so-called "wild optimism," without relying on the global structure of the underlying function class but only assuming black box access to the training algorithm and a single dataset. We begin by generating two sets of artificially modified pseudo-outcomes created by stochastically perturbing the derivatives with carefully chosen scaling. Using these pseudo-labeled datasets, we refit the black-box procedure twice to obtain two wild predictors and derive an efficient excess risk upper bound under the fixed design setting. Requiring no prior knowledge of the complexity of the underlying function class, our method is essentially model-free and holds significant promise for theoretically evaluating modern opaque deep neural networks and generative models, where traditional learning theory could be infeasible due to the extreme complexity of the hypothesis class.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。