用进化算法自动搜索最优删数据损失函数,让模型更精准地删掉指定数据。
EvoMU: Evolutionary Machine Unlearning
- 通过进化搜索在海量损失函数中自动找适合特定任务的删数据方案。
- 在4个数据集上超越现有方法,最高提升12.3%性能,仅用40亿参数模型。
- 无需人工干预,适合资源有限却想高效实现数据删除的研究者。
机器删数据旨在移除特定训练数据(如敏感或版权内容)。主流方法是用删数据损失微调已有模型以保持整体性能。但合适的损失函数空间巨大,寻找最优函数极具挑战;且不同数据结构和重叠情况可能导致同一损失在某些场景下过删或欠删。本文提出EvoMU,采用进化搜索机制,在庞大损失函数空间中自动发现任务特异性损失。该方法无需人工参与,可找到优于文献中现有损失的方案。本工作是自动科学发现的实例,即AI协作者。与以往工作不同,我们仅用小型40亿参数模型(Qwen3-4B-Thinking)就达到最先进水平,证明了低成本AI协作者的潜力。实验表明,我们在TOFU-5%、TOFU-10%、MUSE和WMDP数据集上均超越现有基于损失的删数据方法,通过合成新型删数据损失实现。代码已公开于https://github.com/Batorskq/EvoMU。
原文摘要 · Abstract (English)
Machine unlearning aims to unlearn specified training data (e.g. sensitive or copyrighted material). A prominent approach is to fine-tune an existing model with an unlearning loss that retains overall utility. The space of suitable unlearning loss functions is vast, making the search for an optimal loss function daunting. Additionally, there might not even exist a universally optimal loss function: differences in the structure and overlap of the forget and retain data can cause a loss to work well in one setting but over-unlearn or under-unlearn in another. Our approach EvoMU tackles these two challenges simultaneously. An evolutionary search procedure automatically finds task-specific losses in the vast space of possible unlearning loss functions. This allows us to find dataset-specific losses that match or outperform existing losses from the literature, without the need for a human-in-the-loop. This work is therefore an instance of automatic scientific discovery, a.k.a. an AI co-scientist. In contrast to previous AI co-scientist works, we do so on a budget: We achieve SotA results using a small 4B parameter model (Qwen3-4B-Thinking), showing the potential of AI co-scientists with limited computational resources. Our experimental evaluation shows that we surpass previous loss-based unlearning formulations on TOFU-5%, TOFU-10%, MUSE and WMDP by synthesizing novel unlearning losses. Our code is available at https://github.com/Batorskq/EvoMU.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。