提出预审计方法，提前预测大模型删知识时的副作用。
PreUnlearn: Auditing Collateral Knowledge Damage Before Large Language Model Unlearning
- 从数据角度分析删知识时影响扩散规律
- 发现副作用随语义距离减弱但不消失
- 利用数据特征预判删除风险，适合安全可控删知识场景
大语言模型的机器删知识旨在移除特定知识同时保留其余能力。然而，待删与需保留的知识边界常不清晰，相关甚至远距离信息可能在模型中纠缠。本文从数据视角研究删知识效应，测量删知识影响从目标集向同域及跨域知识的传播。结果发现：副作用强度在靠近目标集处最强，随语义距离减弱，但在领域边界不消失。进一步提出删知识前审计任务，分析哪些数据特征最能预测下游损害。结果显示，目标集与评估集间交互特征提供最强信号，表明副作用在模型更新前已反映在数据几何中。该发现使删知识审计成为早期预警工具，有助于识别高风险删知识操作并设计更可靠的删知识流程。
原文摘要 · Abstract (English)
Machine unlearning for large language models (LLMs) aims to remove specified knowledge while preserving the rest of the model's capabilities. However, the boundary between knowledge to forget and knowledge to retain is often unclear, since related and even distant information may be entangled in the model. In this paper, we study LLM unlearning from a data-centric perspective and measure how unlearning effects propagate from the forget set to same-domain and distant-domain knowledge. We find a consistent decay pattern: collateral damage is strongest near the forget set, weakens with semantic distance, but does not disappear at domain boundaries. We further ask whether such damage can be audited before unlearning is executed. We formulate forget-set auditing as a pre-unlearning prediction task and analyze which data features are most predictive of downstream damage. Our results show that interaction features between the forget set and evaluation set provide the strongest signals, suggesting that collateral damage is partly reflected in data geometry before model updates occur. These findings position forget-set auditing as an early warning tool for identifying risky unlearning runs and designing more reliable unlearning procedures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。