用分层隐私保护机制，防止大模型知识泄露。
AlignDP: Hybrid Differential Privacy with Rarity-Aware Protection for LLMs
- 区分罕见与常见数据字段，分别采用PAC不可区分和RAPPOR加噪
- 罕见字段实现近零ε的本地差分隐私，常见字段保持频率估计无偏
- 适合关注模型隐私防护的研究者与实际部署人员
大语言模型面临知识提取、模型蒸馏和未经授权微调的风险。现有防御手段如水印或监控均在泄露后才起作用。我们设计了AlignDP，一种在数据接口处阻断知识迁移的混合隐私保护机制。核心思路是将数据分为罕见字段与非罕见字段：罕见字段通过PAC不可区分性防护，实现有效的零ε局部差分隐私；非罕见字段则使用RAPPOR技术，在局部差分隐私下保持无偏频率估计。全局聚合器负责组合隐私预算与保证。该双层设计既能隐藏罕见事件，又能对高频事件添加可控噪声。我们证明了PAC扩展至全局聚合的局限性，给出了RAPPOR估计的边界，并分析了效用权衡。小型模拟验证了可行性：罕见类别保持隐蔽，高频类别可恢复且误差小。
原文摘要 · Abstract (English)
Large language models are exposed to risks of extraction, distillation, and unauthorized fine-tuning. Existing defenses use watermarking or monitoring, but these act after leakage. We design AlignDP, a hybrid privacy lock that blocks knowledge transfer at the data interface. The key idea is to separate rare and non-rare fields. Rare fields are shielded by PAC indistinguishability, giving effective zero-epsilon local DP. Non-rare fields are privatized with RAPPOR, giving unbiased frequency estimates under local DP. A global aggregator enforces composition and budget. This two-tier design hides rare events and adds controlled noise to frequent events. We prove limits of PAC extension to global aggregation, give bounds for RAPPOR estimates, and analyze utility trade-off. A toy simulation confirms feasibility: rare categories remain hidden, frequent categories are recovered with small error.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。