解决大规模模型编辑时嵌入错位问题，提升知识更新可靠性
EAMET: Robust Massive Model Editing via Embedding Alignment Optimization
- 通过对齐关键与残差嵌入空间，实现稳定的大规模编辑
- 在10,000条事实编辑下仍保持约90%的准确率
- 适合需要频繁更新知识的复杂应用场景
模型编辑技术对高效更新大语言模型（LLMs）的知识至关重要。然而，现有方法在大规模编辑场景中性能下降，尤其在实际评估指标下表现不佳，且在上下文丰富或同一主题多条事实同时编辑时鲁棒性不足。我们归因于知识项间嵌入错位，损害了大规模编辑的可靠性。为此，提出EAMET（Embedding Alignment Model Editing in Transformers），通过对齐关键与残差嵌入空间来解决该问题。在六种LLM和三个数据集上的大量实验表明，EAMET始终优于现有方法，在编辑10,000条事实时达到约90%的编辑效能。代码与数据集已公开于https://ybdai7.github.io/eamet-page/。
原文摘要 · Abstract (English)
Model editing techniques are essential for efficiently updating knowledge in large language models (LLMs). However, the effectiveness of existing approaches degrades in massive editing scenarios, particularly when evaluated with practical metrics. Their robustness is also limited in context-rich settings or when editing multiple facts of the same subject simultaneously. We attribute these failures to the embedding misalignment among knowledge items, which undermines editing reliability at scale. To address this, we propose EAMET (Embedding Alignment Model Editing in Transformers), which addresses this issue by aligning the space of key and residual embeddings. Extensive experiments across six LLMs and three datasets demonstrate that EAMET consistently outperforms existing methods, achieving about 90\% editing efficacy when editing 10k facts. Codes and datasets are publicly available at https://ybdai7.github.io/eamet-page/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。