arXiv:2505.09432cs.LGstat.ML2025-05NeurIPS被引 4

提出新方法,让平滑损失在转换时仍保持线性误差边界。

Establishing Linear Surrogate Regret Bounds for Convex Smooth Losses via Convolutional Fenchel-Young Losses

  • 用卷积负熵构造平滑代理损失,结合预测映射实现线性误差界。
  • 首次在任意离散目标损失下实现平滑损失与线性误差的统一。
  • 适合研究优化效率与统计一致性平衡的机器学习学者。

代理误差界(即超额风险界)连接了代理损失与目标损失的收敛速率。若代理误差界为线性,则误差传递无损。尽管凸平滑代理损失因高效估计与优化而受青睐,但学界普遍认为其平滑性与线性误差界存在权衡:经过误差传递后,优化性能不可避免下降。本文通过构建基于卷积负熵生成的Fenchel-Younng损失,克服此困境。该损失等价于广义负熵与目标贝叶斯风险的下确界卷积,既保持损失平滑性,又确保代理误差界线性。此外,下确界卷积还带来对类概率的一致估计。结果展示了凸分析如何深化风险最小化中的优化与统计效率。

原文摘要 · Abstract (English)

Surrogate regret bounds, also known as excess risk bounds, bridge the gap between the convergence rates of surrogate and target losses. The regret transfer is lossless if the surrogate regret bound is linear. While convex smooth surrogate losses are appealing in particular due to the efficient estimation and optimization, the existence of a trade-off between the loss smoothness and linear regret bound has been believed in the community. Under this scenario, the better optimization and estimation properties of convex smooth surrogate losses may inevitably deteriorate after undergoing the regret transfer onto a target loss. We overcome this dilemma for arbitrary discrete target losses by constructing a convex smooth surrogate loss, which entails a linear surrogate regret bound composed with a tailored prediction link. The construction is based on Fenchel--Young losses generated by the convolutional negentropy, which are equivalent to the infimal convolution of a generalized negentropy and the target Bayes risk. Consequently, the infimal convolution enables us to derive a smooth loss while maintaining the surrogate regret bound linear. We additionally benefit from the infimal convolution to have a consistent estimator of the underlying class probability. Our results are overall a novel demonstration of how convex analysis penetrates into optimization and statistical efficiency in risk minimization.

凸优化误差界损失函数

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。