用软提示蒸馏让小模型安全运行，适合手机等设备部署。
Distilling Safe LLM Systems via Soft Prompts for On Device Settings
- 用软提示+知识蒸馏，把大模型的安全能力迁移到小模型。
- 在多个测试中，安全性和可用性平衡优于其他方法。
- 只需极少额外算力，适合资源受限的终端设备使用。
在资源受限的边缘设备上部署安全的大语言模型（LLM）面临严峻挑战：虽然结合大模型与防护模型的双模型系统能提供有效安全保障，但其巨大的内存和计算开销使其难以在设备端部署。本文对参数高效的安全对齐方法在资源受限场景下的应用进行了全面研究。通过在多种LLM架构、训练目标及参数高效微调方法上的系统评估，我们发现软提示结合基于蒸馏的训练方法始终优于其他方案。我们提出了基于总变差和KL散度的蒸馏框架，能够将防护模型的安全行为有效迁移至学习到的软提示中。在多个基准测试上的评估表明，该组合在安全-效用权衡上优于LoRA适配器、引导向量和直接优化方法，且推理时仅需极少量额外内存与计算。这些发现确立了软提示蒸馏是设备端LLM安全对齐的首选方案。
原文摘要 · Abstract (English)
Deploying safe large language models (LLMs) on resource-constrained edge devices presents a critical challenge: while dual-model systems combining LLMs with guard models provide effective safety guarantees, their substantial memory and computational demands make them prohibitively expensive for on-device deployment. This paper presents a comprehensive study of parameter-efficient safety alignment methods for resource-constrained settings. Through systematic evaluation across multiple LLM architectures, training objectives, and parameter-efficient fine-tuning approaches, we identify that soft prompts combined with distillation-based training consistently outperform alternative methods. We introduce distillation frameworks based on total variation and KL divergence that effectively transfer safety behaviors from guard models into learned soft prompts. Our evaluations on various benchmarks demonstrate that this combination achieves superior safety-usefulness trade-offs compared to LoRA adapters, steering vectors, and direct optimization methods, while requiring minimal additional memory and compute at inference time. These findings establish soft prompt distillation as the preferred approach for safety alignment in on-device LLM deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。