提升大模型知识编辑的稳定性，让修改更可靠。
Keys to Robust Edits: from Theoretical Insights to Practical Advances
- 分离编辑关键词与模型内部表示，避免干扰
- 动态调整关键词，实现稳定与精准的平衡
- 适用于多种模型和编辑方法，效果显著
大语言模型因参数化记忆冲突或过时而难以保持知识准确。现有定位-编辑方法依赖模型内部表征，导致长上下文推理和改写查询中鲁棒性不足。我们发现根本问题：现有语义键无法同时满足鲁棒性（上下文无关激活）与特异性（精确知识区分）。通过理论误差界分析，确立有效编辑的标准。提出可即插即用的鲁棒编辑路径（REP）模块：(1) 将编辑键与原生模型表示解耦；(2) 通过对比学习动态调整键，实现鲁棒性与特异性的平衡。在多种编辑方法（ROME/MEMIT/R-ROME/EMMET）、模型（LLaMA2, QWen, Mistral）和数据集（CounterFact, ZsRE）上的实验表明，REP在鲁棒性测试中成功率最高提升66.4%，且原有成功率不受影响。
原文摘要 · Abstract (English)
Large language models (LLMs) struggle with maintaining accurate knowledge due to conflicting/outdated parametric memories. While locate-and-edit methods address this, their reliance on models' internal representations leads to robustness failures in long-context reasoning and paraphrased queries. We identify a fundamental limitation of locate-and-edit methods: existing semantic keys (for memory localization) cannot simultaneously satisfy robustness (context-invariant activation) and specificity (precise knowledge discrimination). Through theoretical error-bound analysis, we establish formal criteria for effective editing. Our solution introduces \textit{Robust Edit Pathway (REP)}, a plug-and-play module that: (1) disentangles editing keys from native model representations; (2) dynamically adjusts keys via contrastive learning to achieve robustness-specificity balance. Extensive experiments across various editing methods (ROME/MEMIT/R-ROME/EMMET), existing LLMs (LLaMA2, QWen, Mistral), and datasets (CounterFact, ZsRE) show that REP improves success rate over robustness tests by up-to 66.4\% while maintaining the success rate unaffected. Our code can be found at https://github.com/ElliottYan/RobustKeyEdit .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。