arXiv:2604.16591cs.LGcs.AI2026-04被引 2

提出新方法提升大模型删记忆效率,解决遗忘与保留的平衡难题。

Randomized Antipodal Search Done Right for Data Pareto Improvement of LLM Unlearning

论文配图:Randomized Antipodal Search Done Right for Data Pareto Improvement of LLM Unlearning
图 1 · 摘自论文原文
  • 用随机反向搜索结合哈希技术,高效定位需删除的数据。
  • 在多个模型上实现遗忘率提升且保留能力不下降,效率达次线性复杂度。
  • 适合实际部署中突发性遗忘需求,可推广至各类大模型场景。

大语言模型有时会记住有害知识,需在部署后删除。以往的机器遗忘方法多依赖参数优化,但假设遗忘和保留数据集可直接获取,这在现实中很少成立。遗忘通常由推理时的不良生成触发,因此相关数据的检索成为核心挑战。本文提出数据帕累托改进的概念,形式化了检索如何拓展遗忘与保留之间的最优权衡边界。为此,我们提出RASLIK算法——基于线性影响核的随机反向搜索,结合排列投影哈希,降低选择方差,实现次线性复杂度,并在质量和效率上均获得双重提升。在多种模型、数据集和遗忘算法下,RASLIK持续优于确定性基线甚至理想采样,证明随机搜索是数据驱动遗忘的一种原则性且可扩展的解决方案。

原文摘要 · Abstract (English)

Large language models (LLMs) sometimes memorize undesirable knowledge, which must be removed after deployment. Prior work on machine unlearning has focused largely on optimization methods that adjust parameters to enforce forgetting while preserving retention. However, these approaches assume that the forget and retain sets are readily available, which rarely holds in practice. Unlearning is typically triggered by an undesired generation at inference time, making the retrieval of relevant data the central challenge. We introduce the notion of data Pareto improvement for LLM unlearning, which formalizes how retrieval can expand the achievable trade-off frontier between forgetting and retention. To realize this principle, we propose Randomized Antipodal Search on Linearized Influence Kernel (RASLIK), a retrieval algorithm that combines permutation-projection hashing with randomized antipodal search. RASLIK reduces selection variance, achieves sublinear complexity, and yields a double gain in both quality and efficiency. Across multiple models, datasets, and unlearning algorithms, RASLIK consistently outperforms deterministic baselines and even oracle sampling, establishing randomized search as a principled and scalable solution for data-centric unlearning.

大模型遗忘学习检索算法高效计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。