arXiv:2605.04671cs.LG2026-05被引 2

用信息论方法评估样本可信度,提升梯度提升在噪声标签下的稳定性。

ITBoost: Information-Theoretic Trust for Robust Boosting

  • 基于残差轨迹复杂度衡量样本可信度,拒绝异常波动的样本
  • 在多个表格数据集上噪声环境下优于主流提升与深度模型
  • 理论证明其泛化界更紧,且干净数据上性能不下降

梯度提升在表格数据学习中仍具强大能力,但标签噪声会显著降低其性能。这主要源于算法对大梯度样本的过度关注,而未区分错误是来自有信息量的难例还是不可靠标签。本文提出信息论可信度提升(ITBoost),通过分析样本残差随迭代的演化轨迹,利用最小描述长度原则衡量残差路径复杂度。残差模式不规则的样本被视为不可信,自动降权。理论上,我们在标签噪声下推导出更紧的泛化界。实验表明,ITBoost在多种表格基准上均在噪声环境中优于领先提升与深度表格模型,同时在干净数据上保持最优平均性能。

原文摘要 · Abstract (English)

Gradient boosting remains a strong and widely used method for tabular data learning, but its performance often degrades when training labels are noisy. This behavior is largely related to the way boosting algorithms emphasize samples with large gradients, without explicitly accounting for whether such errors originate from informative hard cases or from unreliable labels. We address this issue by reconsidering how sample reliability is evaluated during boosting. Instead of relying on instantaneous error, we examine the evolution of each sample's residuals across iterations. Based on this insight, we propose Information-Theoretic Trust Boosting (ITBoost), which uses the Minimum Description Length principle to measure the complexity of residual trajectories. Samples whose residual patterns fluctuate in an irregular manner are treated as less trustworthy and are down-weighted during learning. Theoretically, we derive a tighter generalization bound for ITBoost under label noise. Empirical results on various tabular benchmarks indicate that ITBoost provides improved robustness in noisy environments over leading boosting and deep tabular models, while retaining best average performance on clean data.

梯度提升噪声鲁棒信息论

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。