提出一种新视角求解带f散度正则的损失最小化问题。
A Dual Optimization View to Empirical Risk Minimization with f-Divergence Regularization
- 用对偶优化和变分法重构正则化问题
- 导出归一化函数的非线性微分方程表达式
- 在弱条件下实现高效计算，适合理论研究者
本文引入了带f散度正则的经验风险最小化（ERM-fDR）的对偶形式。通过对偶优化问题的解与作为隐函数的归一化函数相联系，利用Legendre-Fenchel变换和隐函数定理，推导出归一化函数的非线性常微分方程表达式。该方程及其性质在温和条件下提供了计算ERM-fDR解中归一化函数的高效方法，为相关模型训练提供理论支持。
原文摘要 · Abstract (English)
The dual formulation of empirical risk minimization with f-divergence regularization (ERM-fDR) is introduced. The solution of the dual optimization problem to the ERM-fDR is connected to the notion of normalization function introduced as an implicit function. This dual approach leverages the Legendre-Fenchel transform and the implicit function theorem to provide a nonlinear ODE expression to the normalization function. Furthermore, the nonlinear ODE expression and its properties provide a computationally efficient method to calculate the normalization function of the ERM-fDR solution under a mild condition.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。