加权经验风险最小化可提升特定区域的预测性能。
Reweighting Improves Conditional Risk Bounds
- 引入数据相关权重函数优化经验风险最小化。
- 在满足平衡伯恩斯坦条件时，误差界中出现更优的数据依赖常数项。
- 适用于分类中的大间隔区域或异方差回归中的低方差区域。
本文研究加权经验风险最小化（weighted ERM）方法，该方法在最小化经验风险时引入额外的数据依赖权重函数。在一般‘可平衡’伯恩斯坦条件下，我们设计的加权ERM估计器在某些子区域内表现优于标准ERM，其优势体现在误差界的依赖数据常数项上。这些子区域分别对应分类场景下的大间隔区域和异方差回归场景下的低方差区域。实验结果基于合成数据验证了理论发现。
原文摘要 · Abstract (English)
In this work, we study the weighted empirical risk minimization (weighted ERM) schema, in which an additional data-dependent weight function is incorporated when the empirical risk function is being minimized. We show that under a general ``balanceable" Bernstein condition, one can design a weighted ERM estimator to achieve superior performance in certain sub-regions over the one obtained from standard ERM, and the superiority manifests itself through a data-dependent constant term in the error bound. These sub-regions correspond to large-margin ones in classification settings and low-variance ones in heteroscedastic regression settings, respectively. Our findings are supported by evidence from synthetic data experiments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。