利用参数空间对称性提升优化学习效率，让算法自动发现高效更新策略。
Improving Learning to Optimize Using Parameter Symmetries
- 通过学习对称变换与局部更新联合优化，逼近牛顿法效果。
- 实验表明该方法在基准测试中显著提升性能，加入动量进一步增强效果。
- 适合研究元优化、神经网络训练加速的学者参考。
我们分析了一种利用参数空间对称性提升优化效率的学习型优化（L2O）算法。先前研究表明，联合学习对称变换与局部更新可提升元优化器性能。本文理论证明，即使未找到最优群元素，该方法仍局部表现如牛顿法。此外，我们提供一个例子，证明算法在训练中可准确学习到正确的对称变换。为实证评估带跳跃（teleportation）的L2O，我们构建了一个基准测试，分析其成功与失败案例，并发现动量等改进措施能进一步提升性能。结果表明，利用神经网络参数空间对称性是推动元优化发展的有效途径。
原文摘要 · Abstract (English)
We analyze a learning-to-optimize (L2O) algorithm that exploits parameter space symmetry to enhance optimization efficiency. Prior work has shown that jointly learning symmetry transformations and local updates improves meta-optimizer performance. Supporting this, our theoretical analysis demonstrates that even without identifying the optimal group element, the method locally resembles Newton's method. We further provide an example where the algorithm provably learns the correct symmetry transformation during training. To empirically evaluate L2O with teleportation, we introduce a benchmark, analyze its success and failure cases, and show that enhancements like momentum further improve performance. Our results highlight the potential of leveraging neural network parameter space symmetry to advance meta-optimization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。