通过删减训练数据提升罕见事件预测,效果优于传统方法。
PARIS: Pruning Algorithm via the Representer theorem for Imbalanced Scenarios
- 基于表示定理计算删样本对验证损失的影响,无需重训练。
- 真实气象数据中删减75%样本,仍保持甚至降低均方误差。
- 适合需要可解释性与高效性的罕见事件回归任务。
不平衡回归的挑战在于,标准经验风险最小化(ERM)会使模型偏向高频数据区域,导致对罕见但高影响的“尾部”事件性能严重下降。现有方法如损失加权或合成过采样常引入噪声、扭曲分布或增加算法复杂度。我们提出PARIS(基于表示定理的不平衡场景剪枝算法),一种通过优化训练集本身来缓解不平衡的原理性框架。PARIS利用神经网络的表示定理,计算出闭式表示删除残差,精确量化移除单个训练样本对验证损失的影响,且无需重新训练。结合高效的Cholesky秩一更新方案,实现快速迭代剪枝,剔除无信息或损害性能的样本。在真实空间天气数据上,PARIS可将训练集减少高达75%,同时保持或改善整体均方根误差(RMSE),优于加权、合成过采样和增强基线方法。结果表明,表示引导的数据集剪枝是一种强大、可解释且计算高效的罕见事件回归方法。
原文摘要 · Abstract (English)
The challenge of \textbf{imbalanced regression} arises when standard Empirical Risk Minimization (ERM) biases models toward high-frequency regions of the data distribution, causing severe degradation on rare but high-impact ``tail'' events. Existing strategies uch as loss re-weighting or synthetic over-sampling often introduce noise, distort the underlying distribution, or add substantial algorithmic complexity. We introduce \textbf{PARIS} (Pruning Algorithm via the Representer theorem for Imbalanced Scenarios), a principled framework that mitigates imbalance by \emph{optimizing the training set itself}. PARIS leverages the representer theorem for neural networks to compute a \textbf{closed-form representer deletion residual}, which quantifies the exact change in validation loss caused by removing a single training point \emph{without retraining}. Combined with an efficient Cholesky rank-one downdating scheme, PARIS performs fast, iterative pruning that eliminates uninformative or performance-degrading samples. We use a real-world space weather example, where PARIS reduces the training set by up to 75\% while preserving or improving overall RMSE, outperforming re-weighting, synthetic oversampling, and boosting baselines. Our results demonstrate that representer-guided dataset pruning is a powerful, interpretable, and computationally efficient approach to rare-event regression.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。