无需重训练,通过模块化滤波实现高效分类模型删减。
MPRU: Modular Projection-Redistribution Unlearning as Output Filter for Classification Pipelines
- 将分类训练视为顺序学习过程,反向撤销最后类别训练。
- 在模型末端添加投影重分配层,实现近似重训练效果。
- 无需原始数据或模型,适合集成到现有分类系统中。
机器遗忘(MU)传统方法多聚焦理论推导与优化目标,但在实际部署中常因需完整访问原始数据集和模型而面临可扩展性问题。本文提出一种新思路:将分类训练视为类别按序学习的归纳过程,遗忘操作可通过逆转最后一个训练阶段实现。具体方法是在模型末尾添加一个投影-重分配层作为输出滤波器,无需访问原始数据或模型即可完成知识删除。该方法具备模块化、模型无关特性,可无缝嵌入现有分类流水线,改动极小。我们在图像数据集(使用CNN的CIFAR-10/100)和表格数据集(使用树模型的Covertype)上进行了多组实验,结果表明其输出性能接近全量重训练模型,同时计算开销显著降低。该方案在保持输出质量的同时,提升了实用性、可扩展性与系统兼容性。
原文摘要 · Abstract (English)
As a new and promising approach, existing machine unlearning (MU) works typically emphasize theoretical formulations or optimization objectives to achieve knowledge removal. However, when deployed in real-world scenarios, such solutions typically face scalability issues and have to address practical requirements such as full access to original datasets and model. In contrast to the existing approaches, we regard classification training as a sequential process where classes are learned sequentially, which we call \emph{inductive approach}. Unlearning can then be done by reversing the last training sequence. This is implemented by appending a projection-redistribution layer in the end of the model. Such an approach does not require full access to the original dataset or the model, addressing the challenges of existing methods. This enables modular and model-agnostic deployment as an output filter into existing classification pipelines with minimal alterations. We conducted multiple experiments across multiple datasets including image (CIFAR-10/100 using CNN-based model) and tabular datasets (Covertype using tree-based model). Experiment results show consistently similar output to a fully retrained model with a high computational cost reduction. This demonstrates the applicability, scalability, and system compatibility of our solution while maintaining the performance of the output in a more practical setting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。