arXiv:2602.02877cs.LG2026-02中稿 · ICML

提出新算法SCENT,高效优化复杂风险最小化问题。

A Geometry-Aware Efficient Algorithm for Compositional Entropic Risk Minimization

  • 基于双目标优化与随机近端镜像下降,捕捉目标几何结构
  • 理论证明收敛率达O(1/√T),优于传统方法
  • 在极端分类等任务中稳定超越现有基线,适合高风险建模场景

本文研究一类称为组合式熵风险最小化的问题,其中每个数据的损失由对数期望指数(Log-E-Exp)函数表示。该形式是当对数内显式求和涉及海量项而计算昂贵时,对对数求和指数(LogSumExp)的抽象。尽管此类熵风险目标出现在众多机器学习问题中,现有优化算法仍存在非收敛、数值不稳定及收敛慢等根本缺陷。为此,我们提出一种几何感知的随机算法SCENT,用于将熵风险最小化转化为双极小优化问题的对偶形式。核心在于对偶变量采用由负指数函数诱导的Bregman散度的随机近端镜像下降(SPMD)更新,精确捕捉目标几何结构。主要贡献有三:(i) 证明了凸问题下SCENT算法具有O(1/√T)的收敛率;(ii) 理论揭示了SPMD相比标准SGD在优化对偶变量上的优势;(iii) 在极端分类、部分AUC最大化、对比学习与分布鲁棒优化任务上实证表明,SCENT始终优于现有基线。代码见https://github.com/Optimization-AI/SCENT。

原文摘要 · Abstract (English)

This paper studies optimization for a family of problems termed $\textbf{compositional entropic risk minimization}$, in which each data's loss is formulated as a Log-Expectation-Exponential (Log-E-Exp) function. The Log-E-Exp formulation serves as an abstraction of the Log-Sum-Exponential (LogSumExp) function when the explicit summation inside the logarithm is taken over a gigantic number of items and is therefore expensive to evaluate. While entropic risk objectives of this form arise in many machine learning problems, existing optimization algorithms suffer from several fundamental limitations including non-convergence, numerical instability, and slow convergence rates. To address these limitations, we propose a geometry-aware stochastic algorithm, termed $\textbf{SCENT}$, for the dual formulation of entropic risk minimization cast as a min--min optimization problem. The key to our design is a $\textbf{stochastic proximal mirror descent (SPMD)}$ update for the dual variable, equipped with a Bregman divergence induced by a negative exponential function that faithfully captures the geometry of the objective. Our main contributions are threefold: (i) we establish an $O(1/\sqrt{T})$ convergence rate of the proposed SCENT algorithm for convex problems; (ii) we theoretically characterize the advantages of SPMD over standard SGD update for optimizing the dual variable; and (iii) we demonstrate the empirical effectiveness of SCENT on extreme classification, partial AUC maximization, contrastive learning and distributionally robust optimization, where it consistently outperforms existing baselines. Code is available at https://github.com/Optimization-AI/SCENT.

优化算法风险最小化随机优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。