按数据点粒度评估删去难度，提升模型删数效率
Leveraging Per-Instance Privacy for Machine Unlearning
- 用每个数据点的隐私损失替代整体最差情况，更精准衡量删去难度
- 理论预测在SGLD和普通微调中均成立，且与数据难易度高度相关
- 可识别难删数据组，适合需要精细化删数的场景
我们提出一种基于个体数据点的机器删数方法，通过将最坏情况下的隐私损失替换为每个数据点的局部隐私损失（Thudi et al., 2024），以更精确地量化删去某条数据的难度。该方法基于对噪声梯度下降的分析改进（Chien et al., 2024），利用瑞尼散度衡量重训练时移除单个数据点的影响。实验表明，该理论在随机梯度朗之万动力学（SGLD）及标准微调中均有效。此外，个体隐私损失与多种现有数据难易度指标高度相关，并能识别出更难删的数据子集。我们还提出了基于损失屏障的新评估方法。结果为个性化、高效且自适应的删数策略提供了理论基础。
原文摘要 · Abstract (English)
We present a principled, per-instance approach to quantifying the difficulty of unlearning via fine-tuning. We begin by sharpening an analysis of noisy gradient descent for unlearning (Chien et al., 2024), obtaining a better utility-unlearning tradeoff by replacing worst-case privacy loss bounds with per-instance privacy losses (Thudi et al., 2024), each of which bounds the (Renyi) divergence to retraining without an individual data point. To demonstrate the practical applicability of our theory, we present empirical results showing that our theoretical predictions are born out both for Stochastic Gradient Langevin Dynamics (SGLD) as well as for standard fine-tuning without explicit noise. We further demonstrate that per-instance privacy losses correlate well with several existing data difficulty metrics, while also identifying harder groups of data points, and introduce novel evaluation methods based on loss barriers. All together, our findings provide a foundation for more efficient and adaptive unlearning strategies tailored to the unique properties of individual data points.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。