提出无需雅可比矩阵的根查找优化方法，解决强化学习与生成模型中的方差陷阱问题。
Escaping the Variance Trap: Jacobian-Free Dynamics for Root-Finding Bilevel Optimization
- 直接沿根误差更新，避开雅可比计算和方差放大
- 在非线性微分方程控制中实现17倍加速，基线方法失败
- 适用于需要稳定熵调节或高质量生成的场景
许多核心机器学习任务，如强化学习中的熵调节和生成对抗网络的均衡，本质上是随机根查找问题而非损失最小化。但通常通过平方残差强行转入最小化框架，引入我们称为“方差陷阱”的关键缺陷。标准双层最小化算法需估计包含隐式雅可比的超梯度，在随机环境下这些项会放大噪声，导致收敛不稳定。本文将根查找双层优化（RF-BO）定义为一类独立问题，并提出基于两时标随机逼近（TTSA）的雅可比自由解法，直接沿根误差更新，从根本上避免方差放大。我们在马尔可夫噪声下首次提供TTSA的非渐近收敛保证。大量实验表明该范式具有决定性优势：相比平方残差与隐式梯度基线，其在SimCLR中提升2.6%的top-1准确率，在非线性微分方程控制中实现17倍更快收敛（基线失败），强化学习中显著改善熵稳定性，生成建模中质量提升11.1%。
原文摘要 · Abstract (English)
Many central machine learning tasks, from entropy tuning in reinforcement learning to equilibrating generative adversarial networks, are fundamentally stochastic root-finding problems rather than loss minimization. Yet, they are frequently forced into a minimization framework via squared residuals, introducing a critical flaw we identify as the Variance Trap. Standard bilevel minimization algorithms require estimating hypergradients involving implicit Jacobians; in stochastic settings, these terms act as noise amplifiers, destabilizing convergence. We formalize Root-Finding Bilevel Optimization (RF-BO) as a distinct problem class that bypasses this pathology. We propose a Jacobian-free solution using Two-Time-Scale Stochastic Approximation (TTSA) that updates directly along the root error, structurally avoiding variance amplification. We provide the first non-asymptotic convergence guarantees for TTSA in this setting under Markovian noise. Extensive experiments demonstrate the decisive advantage of this paradigm: compared to squared-residual and implicit-gradient baselines, our framework achieves a 2.6\% top-1 accuracy gain in SimCLR, 17$\times$ faster convergence in non-linear ODE control where baselines fail, significantly improved entropy stability in reinforcement learning, and an 11.1\% quality improvement in generative modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。