在双曲空间优化扩散模型，加速训练初期收敛。
Hyperbolic Optimization
- 将Adam优化器扩展至双曲空间，支持Poincaré球上的学习。
- 在特定数据集上实现更快收敛，生成质量不下降。
- 适合需快速训练的双曲嵌入任务，如层级数据建模。
本文研究双曲流形上的优化方法。基于黎曼优化原理，将双曲随机梯度下降（Riemannian SGD的特例）扩展为双曲Adam优化器。该方法尤其适用于Poincaré球上的学习，也可能在欧氏及其他非欧环境中带来收益，因其促使学习到Poincaré嵌入表示。这种表示在训练早期参数远离最优时能加速收敛。作为案例研究，我们使用双曲时间离散化的朗之万动力学训练扩散模型，结果表明在某些数据集上可实现更快收敛，同时保持生成质量。
原文摘要 · Abstract (English)
This work explores optimization methods on hyperbolic manifolds. Building on Riemannian optimization principles, we extend the Hyperbolic Stochastic Gradient Descent (a specialization of Riemannian SGD) to a Hyperbolic Adam optimizer. While these methods are particularly relevant for learning on the Poincaré ball, they may also provide benefits in Euclidean and other non-Euclidean settings, as the chosen optimization encourages the learning of Poincaré embeddings. This representation, in turn, accelerates convergence in the early stages of training, when parameters are far from the optimum. As a case study, we train diffusion models using the hyperbolic optimization methods with hyperbolic time-discretization of the Langevin dynamics, and show that they achieve faster convergence on certain datasets without sacrificing generative quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。