让大模型遗忘特定知识，比删除数据更高效可行
Lifting Data-Tracing Machine Unlearning to Knowledge-Tracing for Foundation Models
- 用知识追踪替代数据追踪，实现对大模型的定向遗忘
- 可应对监管、企业等多方提出的多样化遗忘需求
- 更贴近人脑遗忘机制，适合需要可控知识的大模型应用
机器遗忘旨在移除特定训练数据及其对模型的影响（如数据所有者撤回授权时）。本文提出将数据追踪型机器遗忘提升至大模型的知识追踪层面。实践中，由于监管机构、企业用户或产品团队无法访问大模型的海量训练数据，难以精准追踪具体数据点，但可明确要求模型遗忘某类知识或能力。认知研究也表明，知识追踪更符合人脑遗忘机制。我们进一步探讨了知识追踪机器遗忘面临的非平凡挑战，并以视觉-语言大模型为例，说明如何实现该范式。代码已公开于：https://1yuwen.github.io/Knowledge-Tracing-MU-Page。
原文摘要 · Abstract (English)
Machine unlearning removes certain training data points and their influence from AI models (e.g., when a data owner revokes their consent to allow models to learn from the data). In this position paper, we propose to lift data-tracing machine unlearning to knowledge-tracing for foundation models (FMs). We support this position based on practical needs and insights from cognitive studies. Practically, tracing data cannot meet the diverse unlearning requests for FMs, which may be from regulators, enterprise users, product teams, etc., who have no access to FMs' massive training data. Instead, it is convenient for these parties to issue an unlearning request about the knowledge or capability FMs (should not) possess. Cognitively, knowledge-tracing unlearning aligns with how the human brain forgets more closely than tracing individual training data points does. We further discuss the nontrivial challenges in the knowledge-tracing machine unlearning paradigm. Finally, we provide a concrete case study about a vision-language FM to illustrate how an unlearner might instantiate the knowledge-tracing machine unlearning paradigm. Code is available at: https://1yuwen.github.io/Knowledge-Tracing-MU-Page.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。