提出可精确删除数据点的算法，保持模型性能不变
Data Deletion for Linear Regression with Noisy SGD
- 基于信噪比筛选可删除数据点，实现无损删除
- 在合成数据集上验证算法有效，模型性能不变
- 为数据效率与隐私保护提供新思路，适合研究者参考
在大数据与机器学习时代，如何在不损害训练效果的前提下缩减训练数据规模，以提升效率至关重要。然而，实际操作中面临两大挑战：如何找到可删除而不显著影响结果的数据点，以及避免因删除导致欠拟合。本文针对经典线性回归任务中的单步噪声随机梯度下降（1-step noisy SGD），提出了‘完美删除点’问题——即寻找一个数据点，使其删除后模型参数与未删除时完全一致。我们引入信噪比（signal-to-noise ratio）作为关键指标，发现其值与完美删除点的选择密切相关。据此设计了一种算法，并在合成数据集上实证验证了其有效性。进一步分析表明，该方法不仅不影响训练性能，还可能优化隐私预算，凸显其在数据效率与隐私保护方面的潜力。本研究强调了数据删除的重要性，呼吁更多相关研究。
原文摘要 · Abstract (English)
In the current era of big data and machine learning, it's essential to find ways to shrink the size of training dataset while preserving the training performance to improve efficiency. However, the challenge behind it includes providing practical ways to find points that can be deleted without significantly harming the training result and suffering from problems like underfitting. We therefore present the perfect deleted point problem for 1-step noisy SGD in the classical linear regression task, which aims to find the perfect deleted point in the training dataset such that the model resulted from the deleted dataset will be identical to the one trained without deleting it. We apply the so-called signal-to-noise ratio and suggest that its value is closely related to the selection of the perfect deleted point. We also implement an algorithm based on this and empirically show the effectiveness of it in a synthetic dataset. Finally we analyze the consequences of the perfect deleted point, specifically how it affects the training performance and privacy budget, therefore highlighting its potential. This research underscores the importance of data deletion and calls for urgent need for more studies in this field.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。