提出在线学习与删除机制，实现数据删除后模型输出无差别。
Online Learning and Unlearning
- 基于在线梯度下降，分被动与主动两种方式实现删除
- 删除后模型输出与从未训练过该数据的模型无统计差异
- 在保证删除效果的同时保持接近标准算法的性能
我们形式化了在线学习-删除问题：模型在序列更新中需支持中间删除请求。数据点被删除后，后续所有输出必须在统计上等同于从未训练过该数据的模型。本文提出两种在线学习-删除算法（OLU），均基于在线梯度下降（OGD）。被动型OLU利用OGD的压缩性质，在删除时注入噪声，无需额外计算；主动型OLU使用离线删除算法，将模型向不含被删数据的解空间移动。在标准凸性与光滑性假设下，两者均达到与标准OGD相当的后悔界，证明了在提供删除保障的同时仍可维持竞争性性能。
原文摘要 · Abstract (English)
We formalize the problem of online learning-unlearning, where a model is updated sequentially in an online setting while accommodating unlearning requests between updates. After a data point is unlearned, all subsequent outputs must be statistically indistinguishable from those of a model trained without that point. We present two online learner-unlearner (OLU) algorithms, both built upon online gradient descent (OGD). The first, passive OLU, leverages OGD's contractive property and injects noise when unlearning occurs, incurring no additional computation. The second, active OLU, uses an offline unlearning algorithm that shifts the model toward a solution excluding the deleted data. Under standard convexity and smoothness assumptions, both methods achieve regret bounds comparable to those of standard OGD, demonstrating that one can maintain competitive regret bounds while providing unlearning guarantees.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。