用模型合并技术精准删除大模型敏感知识，避免删过头或没删干净。
ZJUKLAB at SemEval-2025 Task 4: Unlearning via Model Merging
- 通过TIES-合并将两个专用模型融合，生成平衡的去敏感化模型。
- 在任务总分上得分为0.944，整体聚合得分0.487，排名第二。
- 揭示了现有评估方法的局限性，呼吁更全面的评价体系。
本文介绍浙江大学团队在SemEval-2025 Task 4：从大语言模型中消除敏感内容任务中的提交方案。该任务旨在选择性地擦除大模型中的敏感知识，避免过度遗忘与遗忘不足的问题。我们提出一种基于模型合并（特别是TIES-Merging）的去敏感系统，将两个专用模型融合为一个更均衡的去敏感模型。系统在26支参赛队伍中表现优异，线上得分达0.944（任务总分）和0.487（整体聚合得分），位列第二。文中还进行了本地实验，对去敏感过程进行深入分析，涵盖性能轨迹、损失动态与权重视角，并开展多项补充实验以验证方法有效性。同时，我们指出当前评估指标（如MIA分数和ROUGE基指标）不足以全面衡量去敏感效果，强调未来研究需构建更完善的评估方法并重新思考去敏感目标。代码已开源：https://github.com/zjunlp/unlearn/tree/main/semeval25。
原文摘要 · Abstract (English)
This paper presents the ZJUKLAB team's submission for SemEval-2025 Task 4: Unlearning Sensitive Content from Large Language Models. This task aims to selectively erase sensitive knowledge from large language models, avoiding both over-forgetting and under-forgetting issues. We propose an unlearning system that leverages Model Merging (specifically TIES-Merging), combining two specialized models into a more balanced unlearned model. Our system achieves competitive results, ranking second among 26 teams, with an online score of 0.944 for Task Aggregate and 0.487 for overall Aggregate. In this paper, we also conduct local experiments and perform a comprehensive analysis of the unlearning process, examining performance trajectories, loss dynamics, and weight perspectives, along with several supplementary experiments, to understand the effectiveness of our method. Furthermore, we analyze the shortcomings of our method and evaluation metrics, emphasizing that MIA scores and ROUGE-based metrics alone are insufficient to fully evaluate successful unlearning. Finally, we emphasize the need for more comprehensive evaluation methodologies and rethinking of unlearning objectives in future research. Code is available at https://github.com/zjunlp/unlearn/tree/main/semeval25.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。