低精度量化会悄悄恢复被删除的数据，这篇论文提出新方法防止这种泄露。
DurableUn: Quantization-Induced Recovery Attacks in Machine Unlearning
- 在低比特量化下设计了能恢复遗忘内容的攻击，揭示现有评估漏洞。
- 提出新训练目标，在INT4下仍能保持遗忘效果，准确率损失小。
- 适合关注模型隐私安全与部署鲁棒性的研究者和工程师。
机器遗忘旨在满足GDPR等隐私法规要求，移除特定训练数据。然而现有评估假设遗忘与部署精度一致，忽略了生产级大模型通常以低比特精度部署的事实。本文首次系统研究在NF4+LoRA架构下适配器空间的INT4量化对遗忘鲁棒性的影响，评估七种方法在LLaMA-3-8B-Instruct上对TOFU、MUSE-News和WikiBio-WPU数据集的表现。结果表明：INT8无害，而INT4导致遗忘内容恢复最多达22倍，且随数据集难度增加恶化。我们发现FA-RA-Q-INT4三难困境：无法同时实现强遗忘、高性能与量化鲁棒性。密集帕累托扫描揭示一旦获得鲁棒性，准确率会急剧坍塌，无法通过调参改善。为此，我们提出DURABLEUN-SAF（基于锐度感知的遗忘），采用直通估计器梯度通过INT4舍入操作。该方法是唯一实现(0.047, {BF16, INT8, INT4})稳定实证耐久证书的方法：Q-INT4=0.043±0.002，认证成功率3/3；而SalUn在自身发布参数下仅为1/3。我们呼吁将Q-INT4作为标准评估指标，与遗忘率(FA)和恢复率(RA)并列。
原文摘要 · Abstract (English)
Machine unlearning aims to remove specified training data to satisfy privacy regulations such as GDPR. However, existing evaluations assume identical precision at unlearning and deployment, overlooking that production LLMs are deployed at low-bit precision. We show that INT4 quantization systematically restores forgotten content even when models pass compliance audits at bfloat16 (BF16), we term this the quantization recovery attack (QRA). We conduct the first systematic study of unlearning robustness under adapter-space INT4 quantization in the NF4+LoRA regime, evaluating seven methods on LLaMA-3-8B-Instruct across TOFU, MUSE-News, and WikiBio-WPU. INT8 is benign; INT4 induces recovery of up to 22x, worsening with dataset difficulty. We identify the FA-RA-Q-INT4 trilemma: no method simultaneously achieves strong forgetting, high utility, and quantization robustness. A dense Pareto sweep reveals a sharp phase transition once robustness is achieved, retaining accuracy collapses regardless of further tuning. To address this, we propose DURABLEUN-SAF (Sharpness-Aware Forgetting), a quantization-aware objective using Straight-Through Estimator gradients through INT4 rounding. DURABLEUN-SAF is the only method to achieve a stable empirical (0.047, {BF16, INT8, INT4})- durability certificate: Q-INT4= 0.043 +- 0.002, cert rate= 3/3, versus SalUn's cert rate= 1/3 at its own published hyperparameters. We call for Q-INT4 to be adopted as a standard evaluation metric alongside FA and RA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。