提出REPAIR框架，实现大模型低成本精准编辑且不遗忘旧知识。
REPAIR: Robust Editing via Progressive Adaptive Intervention and Reintegration
- 通过闭环反馈与动态记忆管理，逐步更新模型知识
- 多模型测试中编辑准确率提升10%-30%，遗忘显著减少
- 适合需要持续迭代的大模型应用，如知识库维护
大语言模型的后训练受制于获取新知识或修正错误的成本高昂，以及重训常引发的意外副作用。为解决这些问题，我们提出REPAIR（Robust Editing via Progressive Adaptive Intervention and Reintegration），一种支持精确、低成本模型更新并保留非目标知识的终身编辑框架。REPAIR通过闭环反馈机制与动态内存管理，缓解大规模连续编辑带来的不稳定与冲突。同时，通过频繁的知识融合与强局部性约束，有效克服传统分布无关方法忽略意外涟漪效应的缺陷。实验表明，REPAIR在多个模型家族中将编辑准确率提升10%-30%，显著减少知识遗忘。该工作为构建可靠、可扩展、持续演进的大语言模型提供了稳健框架。
原文摘要 · Abstract (English)
Post-training for large language models (LLMs) is constrained by the high cost of acquiring new knowledge or correcting errors and by the unintended side effects that frequently arise from retraining. To address these issues, we introduce REPAIR (Robust Editing via Progressive Adaptive Intervention and Reintegration), a lifelong editing framework designed to support precise and low-cost model updates while preserving non-target knowledge. REPAIR mitigates the instability and conflicts of large-scale sequential edits through a closed-loop feedback mechanism coupled with dynamic memory management. Furthermore, by incorporating frequent knowledge fusion and enforcing strong locality guards, REPAIR effectively addresses the shortcomings of traditional distribution-agnostic approaches that often overlook unintended ripple effects. Our experiments demonstrate that REPAIR boosts editing accuracy by 10%-30% across multiple model families and significantly reduces knowledge forgetting. This work introduces a robust framework for developing reliable, scalable, and continually evolving LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。