arXiv:2509.15141stat.MLcs.LG2025-09

在线学习中提升抗异常值与公平性,只需一次更新就能实现鲁棒优化。

Benefits of Online Tilted Empirical Risk Minimization: A Case Study of Outlier Detection and Robust Regression

  • 通过移除对数项设计新目标函数,保持倾斜参数的敏感性。
  • 负倾斜显著抑制异常值影响,正倾斜提升少数类召回率且精度几乎不变。
  • 每样本计算开销与标准方法相同,适合实时数据流场景。

经验风险最小化(ERM)是监督学习的基础框架,但主要优化平均性能,常忽略公平性与鲁棒性。倾斜经验风险最小化(TERM)通过引入指数倾斜超参数 $t$,在平均性能与最差情况下的公平性和鲁棒性之间取得平衡。然而,在数据逐个到达的在线或流式设置中,经典 TERM 目标退化为标准 ERM,丧失倾斜敏感性。为此,我们提出一种在线 TERM 公式,去除经典目标中的对数项,无需额外计算或内存开销即可保持倾斜效应。该公式实现了由 $t$ 控制的连续权衡:$t \to 0$ 时为标准 ERM,$t > 0$ 强调公平性,$t < 0$ 增强对异常值的鲁棒性。我们在两个代表性流式任务上验证:带对抗异常值的鲁棒线性回归,以及二分类中的少数类检测。结果表明,负倾斜有效抑制异常值影响,正倾斜提升召回率且对精度影响极小,所有操作每样本计算成本等同于 ERM。在线 TERM 因此在高效单样本学习范式下恢复了经典 TERM 的完整鲁棒-公平谱系。

原文摘要 · Abstract (English)

Empirical Risk Minimization (ERM) is a foundational framework for supervised learning but primarily optimizes average-case performance, often neglecting fairness and robustness considerations. Tilted Empirical Risk Minimization (TERM) extends ERM by introducing an exponential tilt hyperparameter $t$ to balance average-case accuracy with worst-case fairness and robustness. However, in online or streaming settings where data arrive one sample at a time, the classical TERM objective degenerates to standard ERM, losing tilt sensitivity. We address this limitation by proposing an online TERM formulation that removes the logarithm from the classical objective, preserving tilt effects without additional computational or memory overhead. This formulation enables a continuous trade-off controlled by $t$, smoothly interpolating between ERM ($t \to 0$), fairness emphasis ($t > 0$), and robustness to outliers ($t < 0$). We empirically validate online TERM on two representative streaming tasks: robust linear regression with adversarial outliers and minority-class detection in binary classification. Our results demonstrate that negative tilting effectively suppresses outlier influence, while positive tilting improves recall with minimal impact on precision, all at per-sample computational cost equivalent to ERM. Online TERM thus recovers the full robustness-fairness spectrum of classical TERM in an efficient single-sample learning regime.

在线学习鲁棒性异常检测优化算法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。