提出更高效的对数和指数优化方法，提升大规模分布优化的计算效率。
Improved Stochastic Optimization of LogSumExp
- 基于安全KL散度构造保凸保光滑近似，可直接用随机梯度优化
- 在分布鲁棒优化与连续最优传输中显著优于现有基线方法
- 适合处理高维或无限项指数求和的优化问题，如机器学习中的分布建模
对数和指数函数作为KL散度的对偶，在熵正则最优传输（OT）和分布鲁棒优化（DRO）等重要问题中具有核心作用。当对数内指数项数量大或为无穷时，优化面临挑战，因梯度需对每一项求导。本文提出一种保持凸性和光滑性的新近似方法，可通过随机梯度法高效优化。该方法源于对偶空间中KL散度的合理修正，构建出一种新的f-散度——安全KL散度。实验与理论分析表明，在基于对数和指数的随机优化问题（如DRO和连续OT）中，本方法显著优于现有基线。
原文摘要 · Abstract (English)
The LogSumExp function, dual to the Kullback-Leibler (KL) divergence, plays a central role in many important optimization problems, including entropy-regularized optimal transport (OT) and distributionally robust optimization (DRO). In practice, when the number of exponential terms inside the logarithm is large or infinite, optimization becomes challenging since computing the gradient requires differentiating every term. We propose a novel convexity- and smoothness-preserving approximation to LogSumExp that can be efficiently optimized using stochastic gradient methods. This approximation is rooted in a sound modification of the KL divergence in the dual, resulting in a new $f$-divergence called the Safe KL divergence. Our experiments and theoretical analysis of the LogSumExp-based stochastic optimization, arising in DRO and continuous OT, demonstrate the advantages of our approach over existing baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。