Natural GaLore让大模型训练更快更省内存，无需额外开销。
Natural GaLore: Accelerating GaLore for memory-efficient LLM Training and Fine-tuning
- 用Woodbury恒等式高效计算二阶信息，替代AdamW优化器
- 在4个规模的Llama模型上降低困惑度，效果优于原版GaLore
- 比LoRA少30%内存，微调精度超GPT4-Turbo 4%
大模型训练面临数据、权重和优化器状态不断增长带来的内存挑战。尽管并行、梯度检查点和卸载等方法可缓解问题，但受限于硬件常难实现。参数高效微调（PEFT）和GaLore通过近似权重或优化器状态降低内存占用。其中，LoRA虽流行但需全秩初始化；而GaLore支持全参数学习且更省内存。本文提出Natural GaLore，一种可直接替换AdamW的优化器，利用Woodbury恒等式高效应用逆经验费雪信息矩阵到低秩梯度。实验表明，引入二阶信息显著加速收敛，尤其在迭代预算有限时。在C4数据集上对60M、130M、350M和1.1B参数的Llama模型进行预训练，其困惑度显著低于原版GaLore，无额外内存开销。在GLUE基准上微调RoBERTa，Natural GaLore准确率86.05%，优于全微调的86.28%。在TinyAgent框架下微调TinyLlama 1.1B模型进行函数调用任务，准确率达83.09%，远超16位LoRA的80.06%，甚至超越GPT4-Turbo的4%，且内存减少30%。所有代码已开源。
原文摘要 · Abstract (English)
Training LLMs presents significant memory challenges due to growing size of data, weights, and optimizer states. Techniques such as data and model parallelism, gradient checkpointing, and offloading strategies address this issue but are often infeasible due to hardware constraints. To mitigate memory usage, alternative methods like Parameter-Efficient-Fine-Tuning (PEFT) and GaLore approximate weights or optimizer states. PEFT methods, such as LoRA, have gained popularity for fine-tuning LLMs, though they require a full-rank warm start. In contrast, GaLore allows full-parameter learning while being more memory-efficient. This work introduces Natural GaLore, a simple drop in replacement for AdamW, which efficiently applies the inverse Empirical Fisher Information Matrix to low-rank gradients using Woodbury's Identity. We demonstrate that incorporating second-order information speeds up optimization significantly, especially when the iteration budget is limited. Empirical pretraining on 60M, 130M, 350M, and 1.1B parameter Llama models on C4 data demonstrate significantly lower perplexity over GaLore without additional memory overhead. By fine-tuning RoBERTa on the GLUE benchmark using Natural GaLore, we demonstrate significant reduction in gap 86.05% vs 86.28% for full-finetuning. Furthermore, fine-tuning the TinyLlama 1.1B model for function calling using the TinyAgent framework shows that Natural GaLore achieving 83.09% accuracy on the TinyAgent dataset, significantly outperforms 16-bit LoRA at 80.06% and even surpasses GPT4-Turbo by 4%, all while using 30% less memory. All code to reproduce the results are available at: https://github.com/selfsupervised-ai/Natural-GaLore.git
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。