提出可控遗忘损失，提升大模型删敏数据效率。
iShumei-Chinchunmei at SemEval-2025 Task 4: A balanced forgetting and retention multi-task framework using effective unlearning loss
- 设计有效遗忘损失，实现精准控制遗忘程度。
- 在基准测试中达成5项排名，平衡遗忘与保留能力。
- 适合需隐私保护的LLM场景，如合规性训练。
随着大语言模型（LLM）广泛应用，如何消除其预训练中记忆的不合规数据成为焦点。机器遗忘旨在有限计算资源下高效擦除敏感信息。为推动该领域研究，SemEval 2025 Task 4设立了三个遗忘数据集，并通过评估遗忘效果与标准能力保留情况建立基准。本文提出一种更可控的遗忘损失——有效遗忘损失（Effective Unlearning Loss），并探索其与多种技术的融合，以实现更高效、更可控的遗忘。所提系统在竞赛排行榜上位列第5。
原文摘要 · Abstract (English)
As the Large Language Model (LLM) gains widespread adoption, increasing attention has been given to the challenge of making LLM forget non-compliant data memorized during its pre-training. Machine Unlearning focuses on efficiently erasing sensitive information from LLM under limited computational resources. To advance research in this area, SemEval 2025 Task 4: "Unlearning Sensitive Content from Large Language Models" introduces three unlearning datasets and establishes a benchmark by evaluating both forgetting effectiveness and the preservation of standard capabilities. In this work, we propose a more controllable forgetting loss, Effective Unlearning Loss, and explore its integration with various techniques to achieve more efficient and controlled unlearning. Our system ultimately ranked 5th on the competition leaderboard.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。