提出在线机器遗忘新方法，实现高效持续学习与删除。
Mo' Memory, Mo' Problems: Stream-Native Machine Unlearning
- 将批量遗忘转化为在线设定，基于损失与删除能力设计算法
- 达到对数级遗憾（O(ln T)），是首个认证的在线遗忘算法
- 仅需恒定内存，适合生产环境持续数据流
机器遗忘研究通常假设静态独立同分布的训练环境，这在现实中并不存在。现代机器学习流水线需要在生产数据流中持续学习、遗忘与预测。本文通过引入后悔、样本复杂度和删除容量的概念，将批量遗忘推广到在线场景。我们首次将后悔上界收紧至对数级（O(ln T)），实现了首个具备证明保障的在线遗忘算法。当采用在线版L-BFGS优化时，该算法在保持恒定内存占用的前提下，达到了当前最优的后悔表现。这一改进显著延长了模型寿命，减少了昂贵的重训练频率，使机器遗忘过程更加高效。
原文摘要 · Abstract (English)
Machine unlearning work assumes a static, i.i.d training environment that doesn't truly exist. Modern ML pipelines need to learn, unlearn, and predict continuously on production streams of data. We translate batch unlearning to the online setting using notions of regret, sample complexity, and deletion capacity. We tighten regret bounds to a logarithmic $\mathcal{O}(\ln{T})$, a first for a certified unlearning algorithm. When fitted with an online variant of L-BFGS optimization, the algorithm achieves state of the art regret with a constant memory footprint. Such changes extend the lifespan of an ML model before expensive retraining, making for a more efficient unlearning process.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。