用生成正则化实现高效精准的模型知识更新
Better Call SAUL: Fluent and Consistent Language Model Editing with Generation Regularization
- 通过拼接句子与随机事实进行生成正则化
- 在三个基准上优于现有方法，保持生成质量
- 计算开销小，适合实际部署场景
为确保大语言模型具备最新知识，需定期更新。但模型编辑面临挑战：修改特定知识时可能影响无关内容。现有方法通过定位并直接修改相关参数，但计算开销大且缺乏理论支撑；而直接微调会损害模型在无关知识上的表现，并显著降低生成流畅性与一致性。为此，本文提出SAUL，一种基于句子拼接与增强随机事实的生成正则化方法。在三个模型编辑基准上的评估显示，SAUL在保持生成质量的同时，性能超越现有最优方法，且显著降低计算开销。
原文摘要 · Abstract (English)
To ensure large language models contain up-to-date knowledge, they need to be updated regularly. However, model editing is challenging as it might also affect knowledge that is unrelated to the new data. State-of-the-art methods identify parameters associated with specific knowledge and then modify them via direct weight updates. However, these locate-and-edit methods suffer from heavy computational overhead and lack theoretical validation. In contrast, directly fine-tuning the model on requested edits affects the model's behavior on unrelated knowledge, and significantly damages the model's generation fluency and consistency. To address these challenges, we propose SAUL, a streamlined model editing method that uses sentence concatenation with augmented random facts for generation regularization. Evaluations on three model editing benchmarks show that SAUL is a practical and reliable solution for model editing outperforming state-of-the-art methods while maintaining generation quality and reducing computational overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。