统一多种鲁棒学习方法,自动选择最优策略应对数据偏差。
Unification and Optimization of Robust Supervised Learning

- 将多种鲁棒学习方法归纳为四个可分阶段的统一框架。
- 在多个基准上表现优于单一方法,且无需预判主要偏差类型。
- 适合不确定数据问题来源的实践者作为默认方案使用。
现有研究提出了多种鲁棒监督学习方法以应对分布偏移、标签噪声和小样本退化等问题,如分布鲁棒优化、标签平滑、邻近风险最小化和Mixup。然而这些方法通常独立发展,迫使从业者预先选择单一故障模式,而实际任务中主导模式往往不明确。为此,本文沿着三个共同设计轴线组织了广泛的方法,并推导出一个可计算的训练流程,将鲁棒学习分解为四个阶段:参考分布增强、输入空间扰动、标签空间扰动和样本级聚合,每个阶段支持悲观、中性或乐观三种立场。由此形成统一的设计空间,支持联合超参数优化,以组合配置适配具体任务的鲁棒策略。在表格数据、图像和奖励建模等多个基准上,联合优化的表现与各场景最优单方法基线相当,为无法预先判断主导偏差类型的从业者提供了可靠默认方案。
原文摘要 · Abstract (English)
The literature has proposed various robust alternatives to empirical risk minimisation to address failure modes such as distribution shift, label noise and finite-sample degeneracies. Examples include distributionally robust optimization, label smoothing, vicinal risk minimization, and Mixup. However, such approaches are typically developed in isolation, forcing practitioners to commit a priori to a single failure mode even when the dominant mode for the task is unclear. To address this, we organize a broad class of existing methods along three common design axes and derive a tractable training procedure that decomposes robust learning into sequential stages (reference distribution enrichment, input-space perturbation, label-space perturbation, and sample-level aggregation), each with a choice of stance (pessimistic, neutral, or optimistic). This results in a unified design space in which joint hyperparameter optimization can compose and configure robustness strategies suited to the task at hand. Across tabular, image, and reward modeling benchmarks, joint hyperparameter optimization is competitive with the best single-method baseline in each setting, offering a reliable default for practitioners who do not know a priori which failure mode dominates their task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。