用随机裁剪降低大模型私密训练的内存开销。
Efficient DP-SGD for LLMs with Randomized Clipping
- 引入随机裁剪与迹估计技术，降低梯度范数计算内存
- 在长文本任务上保持基线性能，内存节省超50%
- 适合需要隐私保护的大模型微调场景
大型语言模型（LLMs）在包含敏感信息的数据集上训练，差分隐私（DP）是提供可证明隐私保护的主流方法。然而，当前最先进的DP训练方法依赖快速梯度裁剪，其内存开销为$O(B \min\{T^2, d^2\})$，其中$B$为批量大小，$T$为序列长度，$d$为模型宽度，随着模型规模和上下文长度增长，该开销变得不可接受。本文提出新型DP-SGD变体DP-SGD-RC，采用随机裁剪结合随机迹估计（如Hutchinson估计器及其改进版Hutch++），显著降低每样本梯度范数估计的内存占用。我们提供了紧致的隐私分析，表明DP-SGD-RC的噪声乘子可与确定性裁剪相媲美。在长上下文基准测试中对Llama~3.2-1B进行微调，涵盖分类、问答和摘要任务，结果表明其在保持基线性能的同时，大幅降低内存与计算需求。
原文摘要 · Abstract (English)
Large language models (LLMs) are trained on vast datasets that may contain sensitive information. Differential privacy (DP), the de facto standard for formal privacy guarantees, provides a principled framework for training LLMs with provable privacy protection. However, state-of-the-art DP training implementations rely on fast gradient clipping techniques with memory overhead $O(B \min\{T^2, d^2\})$, where $B$ is the batch size, $T$ is the sequence length, and $d$ is the model width. This becomes prohibitive as both model size and context length grow. We propose DP-SGD-RC, a novel variant of DP-SGD with randomized clipping that reduces memory and compute complexity. DP-SGD-RC leverages stochastic trace estimation methods, specifically Hutchinson's estimator[Hutchinson, 1989] and its improved variant, Hutch++[Meyer et al., 2021], to reduce the memory footprint of per-sample gradient norm estimation. We provide a tight privacy analysis showing that DP-SGD-RC achieves noise multipliers competitive with deterministic clipping. Experiments fine-tuning Llama~3.2-1B on long-context benchmarks spanning classification, question answering, and summarization tasks demonstrate that DP-SGD-RC matches baseline utility while significantly reducing memory and compute requirements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。