无需调参的去中心化双层优化算法，自动适应梯度变化
Problem-Parameter-Free Decentralized Bilevel Optimization
- 基于累积梯度范数自适应调整步长，统一更新所有变量
- 理论证明收敛速度达 $ ilde{/mathcal{O}}(1/T)$，媲美调优后最优方法
- 对不同步长配置鲁棒，适合实际部署中缺乏参数先验的场景
去中心化双层优化因其在大规模机器学习中的关键作用而受到广泛关注。然而，现有方法通常依赖平滑性、凸性或通信网络拓扑等先验问题参数来设定合适步长，而实践中这些参数往往未知，导致需大量人工调参。本文提出 AdaSDBO，一种全无问题参数依赖的单循环去中心化双层优化算法。该算法基于累积梯度范数自适应调整步长，同时更新所有变量，动态调节优化进程，无需针对具体问题的人工超参数调优。通过严格的理论分析，我们证明 AdaSDBO 达到 $ ilde{/mathcal{O}}(1/T)$ 的收敛速率，与经过良好调参的先进方法性能相当（仅差多对数因子）。大量数值实验表明，相比现有方法，AdaSDBO 在多种步长配置下均表现优异且具备显著鲁棒性。
原文摘要 · Abstract (English)
Decentralized bilevel optimization has garnered significant attention due to its critical role in solving large-scale machine learning problems. However, existing methods often rely on prior knowledge of problem parameters-such as smoothness, convexity, or communication network topologies-to determine appropriate stepsizes. In practice, these problem parameters are typically unavailable, leading to substantial manual effort for hyperparameter tuning. In this paper, we propose AdaSDBO, a fully problem-parameter-free algorithm for decentralized bilevel optimization with a single-loop structure. AdaSDBO leverages adaptive stepsizes based on cumulative gradient norms to update all variables simultaneously, dynamically adjusting its progress and eliminating the need for problem-specific hyperparameter tuning. Through rigorous theoretical analysis, we establish that AdaSDBO achieves a convergence rate of $\widetilde{\mathcal{O}}\left(\frac{1}{T}\right)$, matching the performance of well-tuned state-of-the-art methods up to polylogarithmic factors. Extensive numerical experiments demonstrate that AdaSDBO delivers competitive performance compared to existing decentralized bilevel optimization methods while exhibiting remarkable robustness across diverse stepsize configurations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。