用合成数据+自蒸馏提升私有语言模型生成质量
Differentially Private Learning Needs Better Model Initialization and Self-Distillation
- 先用小模型合成数据并严格筛选，再做私有微调，最后自蒸馏优化
- 生成文本错误率降84.0%，在所有数据集上78.4%胜过原版私有训练
- 小模型如GPT-2即可用于初始化和蒸馏，适合高效部署
差分隐私SGD（DPSGD）可实现语言模型的隐私保护训练，但常导致性能、多样性与语言质量下降。本文提出DPRefine三阶段方法：首先利用小型预训练语言模型生成数据并经过严格过滤进行模型初始化；其次在私有数据上执行差分隐私微调；最后通过自蒸馏进一步优化输出。该方法显著优于原始DPSGD，在AlpacaEval评测中，其生成结果在所有数据集上被偏好率达78.4%。分析表明，DPRefine将生成文本中的语言错误减少84.0%，有效缓解了语法与拼写错误问题，同时减少了非私有模型常见的幻觉细节与引述误标等不一致性。研究还发现，GPT-2等小型模型在初始化与蒸馏阶段表现良好，凸显其在可扩展、高效部署隐私保护语言模型中的潜力。
原文摘要 · Abstract (English)
Differentially private SGD (DPSGD) enables privacy-preserving training of language models, but often reduces utility, diversity, and linguistic quality. We introduce DPRefine, a three-phase method that initializes a model using data synthesis from a small pre-trained LM with rigorous filtering, applies DP finetuning on private data, and performs self-distillation to refine outputs. This approach significantly outperforms vanilla DPSGD, with AlpacaEval preferring DPRefine's generations in 78.4% of cases across all datasets. Our analysis reveals that DPRefine reduces linguistic errors in generated text by 84.0%, mitigating grammar and spelling errors, commonly associated with DPSGD. It also reduces inconsistencies of non-private models, such as hallucinated details and misattributed quotes. We find that small models like GPT-2 can be effective for initialization and distillation, highlighting their potential in enabling scalable and efficient deployment of privacy-preserving language.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。