通过大学习率实现高效零阶微调，稳定提升大模型训练效果。
SubZero+: Efficient Zeroth-Order LLM Fine-Tuning via Large Learning Rates
- 在层内低秩子空间中多查询估计梯度，降低方差
- 使用自适应子空间优化器，学习率范围扩大3倍以上
- 适合追求高效微调且资源受限的研究者和工程师
零阶（ZO）优化可实现无需反向传播的大语言模型微调，但现有方法因梯度估计方差高，导致收敛不稳定且对学习率敏感。本文提出SubZero+，在三个互补方向改进：(i) 在层特定的低秩子空间中进行多查询梯度估计，降低方差且避免多查询悖论；(ii) 设计子空间Adam优化器，利用子空间内多查询梯度统计进行自适应更新；(iii) 对QR分解构建子空间时引入符号修正，确保投影矩阵服从哈尔分布，消除实现依赖的朝向歧义。在1.3B至32B参数量模型上，覆盖SuperGLUE数据集，全参数与LoRA两种场景下，SubZero+均持续优于已有零阶基线，显著扩大稳定学习率范围，缩小与一阶方法差距，仅增加极少内存开销。
原文摘要 · Abstract (English)
Zeroth-order (ZO) optimization enables backpropagation-free fine-tuning of large language models, but existing ZO methods suffer from high-variance gradient estimators, making convergence unstable and highly sensitive to learning rates. We propose SubZero+, an improved SubZero framework that improves stability in three complementary ways: (i) multi-query gradient estimation within layer-specific low-rank subspaces to reduce variance without exhibiting the multi-query paradox; (ii) a subspace Adam optimizer that performs adaptive updates using in-subspace multi-query gradient statistics; and (iii) a sign correction for QR-based subspace construction to ensure Haar-distributed projection matrices, eliminating implementation-dependent orientation ambiguity. Experiments on models from 1.3B to 32B across SuperGLUE, under both full-parameter tuning and LoRA, show that SubZero+ consistently outperforms prior ZO baselines, enlarges the stable learning-rate range, and narrows the gap to first-order methods with minimal extra memory overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。