arXiv:2606.26783cs.LGcs.CL2026-06

复现发现AlphaEdit在长序列编辑中会失效,理论保证并非绝对。

Reproducibility Study of "AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models"

论文配图:Reproducibility Study of "AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models"
图 1 · 摘自论文原文
  • 通过投影到零空间实现知识编辑,理论上不破坏原有知识。
  • 在原始模型上复现成功,但长序列编辑下性能显著下降。
  • 新模型和多任务评估显示其优势不普适,需谨慎部署。

Fang等(2025)提出基于零空间约束的投影方法AlphaEdit,用于定位-编辑类知识编辑任务,理论上可保证编辑不干扰已有知识,并在LLaMA3、GPT2-XL和GPT-J上报告了显著提升。本文对AlphaEdit进行复现研究,在原实验设置下成功再现其报告指标,但发现流畅性与一致性指标存在偏差。扩展至新模型架构后,发现其优势无法统一推广,根源在于定位-编辑范式对新模型架构的假设不成立。进一步将连续编辑次数大幅增加,发现性能在原报告规模下稳定,但随编辑量上升明显退化,表明零空间投影的遗忘保护是有限而非无条件的。额外在BoolQ、HellaSwag和XSTest三个基准上评估,大规模连续编辑同时损害下游任务能力与安全拒答行为。结果确认AlphaEdit在原范围有效,但其核心理论保障受模型架构与编辑规模影响,具实际部署意义。

原文摘要 · Abstract (English)

Fang et al. (2025) introduced a null-space constrained projection, named AlphaEdit, for locate-then-edit knowledge editing methods, theoretically guaranteeing that edits do not disrupt previously preserved knowledge, and reports substantial gains over existing editing methods on LLaMA3, GPT2-XL, and GPT-J. In this work, we present a reproducibility study of AlphaEdit, reproducing its reported results under the original experimental setup and extending the evaluation along three axes: new model architectures, additional downstream benchmarks, and substantially longer sequential editing horizons. We successfully reproduce AlphaEdit's reported metrics across the original models, though we identify a discrepancy in the reported fluency and consistency metric. Extending AlphaEdit to newer model families, we find that its advantage does not generalize uniformly, which we trace to architectural assumptions in the locate-then-edit paradigm that are violated by these newer models. We further stress-test AlphaEdit's central sequential-editing claim by extending the number of edits well beyond those evaluated in the original paper, and find that performance, which is stable at the originally reported scale, degrades as edits reach a much higher count, indicating that the null-space projection's protection against catastrophic forgetting is bounded rather than unconditional. Finally, we extend evaluation of edited models on three extra benchmarks, namely, BoolQ, HellaSwag, and XSTest, and we find that large-scale sequential editing degrades both general downstream task competence and safety-relevant refusal behavior. Our results confirm that AlphaEdit performs as reported within its original scope, while showing that its core theoretical guarantees are sensitive to model architecture and editing scale in ways that have practical implications for its deployment.

知识编辑语言模型可复现性鲁棒性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。