让大模型个性化不泄露用户数据，删掉代理就能彻底清除痕迹。
Separable Expert Architecture: Toward Privacy-Preserving LLM Personalization via Composable Adapters and Deletable User Proxies
- 用可组合的适配器和用户专属代理分离个人数据与共享模型。
- 删除用户代理后，输出恢复基线水平（KL散度约0.21纳特）。
- 适合需要隐私保护的个性化应用，兼容差分隐私训练。
现有模型训练将用户信息直接嵌入共享权重，导致个体数据删除需重训且不可行。本文提出三层架构：静态基础模型、可组合的领域专家LoRA适配器（塑造行为但不携带用户数据）、以及每个用户的可删除代理对象。在Phi-3.5-mini和Llama-3.1-8B上验证，用户数据影响输出但保持隔离，代理移除后输出回归基线（KL散度约0.21纳特，验证通过率82%-89%），跨用户污染几乎为零。因用户信息从不进入共享权重，该架构从设计上抵御模型反演、成员推断和训练数据提取攻击。将机器遗忘从难以实现的权重编辑问题，转变为确定性删除操作，兼顾个性化与隐私保障，并兼容差分隐私随机梯度下降（DP-SGD）以实现隐私保护的共享模型优化。
原文摘要 · Abstract (English)
Current model training approaches incorporate user information directly into shared weights, making individual data removal computationally infeasible without retraining. This paper presents a three-layer architecture that decouples personal data from shared weights by combining a static base model, composable domain-expert LoRA adapters that shape behavior without imparting user data, and per-user proxy artefacts whose deletion constitutes deterministic unlearning. Evaluation on Phi-3.5-mini and Llama-3.1-8B confirms per-user differentiation in which personal data influences outputs while remaining isolated, verified by a return to baseline after proxy removal (KL divergence of approximately 0.21 nats, 82-89% verification pass rate) and near-zero cross-user contamination. Because user-specific information never enters shared weights, the architecture mitigates model inversion, membership inference, and training-data extraction against shared model components by construction. The approach converts machine unlearning from an intractable weight-editing problem into a deterministic deletion operation that preserves personalization alongside privacy-enhancing guarantees and is compatible with differentially private stochastic gradient descent (DP-SGD) for privacy-preserving shared model improvement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。