用随机线性代数加速大规模优化，支持GPU与自动求导。
GPU-Enabled Large-Scale Optimization Using Randomized Linear Algebra
- 基于随机线性代数设计GPU加速的优化求解器。
- 在岭回归等任务中实现显著速度提升。
- 适合需要高效优化与超参数调优的研究者。
本文介绍 rlaopt，一个基于 PyTorch 的大型优化与科学计算工具包，采用随机数值线性代数（RandNLA）技术。尽管 RandNLA 算法已有显著进展，但极少有实现将 GPU 加速与简洁的问题建模接口结合。rlaopt 通过提供正定线性系统和带约束、正则化的凸经验风险最小化求解器，填补了这一空白。这些求解器利用 RandNLA 加速共轭梯度（NystromPCG）、分裂算子（NysADMM）和随机梯度方法（SAPPHIRE）。此外，rlaopt 包含一种建模语言，允许用户以自然数学语法指定问题，自动检查求解器兼容性并执行所需问题分解。求解器支持迭代过程中的反向传播，适用于超参数调优等应用。在岭回归、有界多项式逻辑回归和有界弹性网任务上的实验表明，随机预处理可提升性能，并验证了 GPU 执行带来的显著加速。该工具包开源，许可证为 Apache，源码见 https://github.com/udellgroup/rlaopt，版本 0.1.0 可在 PyPI 获取。
原文摘要 · Abstract (English)
This paper introduces rlaopt, a PyTorch-based package for large-scale optimization and scientific computing using randomized numerical linear algebra (RandNLA). Despite substantial progress in RandNLA-based algorithms, few implementations combine GPU acceleration with a simple interface for specifying optimization problems. rlaopt addresses this gap by providing GPU-enabled solvers for positive-definite linear systems and convex empirical risk minimization with constraints and regularizers. These solvers use RandNLA to accelerate conjugate gradient (NystromPCG), operator splitting (NysADMM), and stochastic gradient methods (SAPPHIRE). Moreover, rlaopt includes a modeling language that lets users specify problems using natural mathematical syntax. rlaopt automatically checks compatibility with the selected solver and performs the required problem decomposition. The solvers also support differentiation through their iterations, enabling applications such as hyperparameter tuning. Experiments on ridge regression, bounded multinomial logistic regression, and bounded elastic net identify when randomized preconditioning improves performance and demonstrate substantial speedups from GPU execution. The package is open-source under an Apache license, with source code at https://github.com/udellgroup/rlaopt and version 0.1.0 available on PyPI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。