无需锚点,端到端优化权重实现多层知识编辑。
DOW-KE: Anchor-Free Multi-Layer Knowledge Editing via Direct End-to-End Weight Optimization

- 直接反向传播最终编辑目标,联合优化所有层的权重更新。
- 在六组实验中,五组达到最高整体得分和邻域特异性。
- 将保护机制嵌入参数化过程,避免编辑干扰原始知识。
多层定位-编辑方法先在选定层优化目标残差流激活(锚点),再逐层实现为权重更新。该流程优化的是中间表示,但部署的是多层权重更新,其通过真实前向传播的联合效果从未被直接优化:无论锚点如何设置或传播,每次更新都来自局部求解,导致传播中的衰减与失真无法纠正,造成锚点目标与实际编辑间的闭合差距。我们提出 DOW-KE,一种无锚点方法,核心原则是:被优化的必须正是被部署的。DOW-KE 将最终编辑目标反向传播至完整模型,联合优化所有编辑层的更新,使跨层传播与耦合进入每个梯度步。同一原则也决定保护机制的位置:将保护投影嵌入更新参数化,置于计算图内部,使每个梯度作用于实际部署的更新;事后约束会重新打开差距,而受限搜索可确保编辑不触及受保护知识。在两个数据集、三种模型上的大规模顺序编辑实验中,DOW-KE 在六组评估设置中,有五组取得最高整体得分与邻域特异性。
原文摘要 · Abstract (English)
Multi-layer locate-then-edit methods for knowledge editing first optimize target residual-stream activations (anchors) at selected layers, then realize them layer by layer as weight updates. This pipeline optimizes an intermediate representation but deploys multi-layer weight updates whose joint effect through the true forward pass is never itself optimized: regardless of how anchors are set or propagated, each update comes from a local solve, so propagation-induced attenuation and distortion go uncorrected, leaving a closure gap between anchor targets and realized edits. We propose DOW-KE, an anchor-free method built on a single principle: what is optimized must be exactly what is deployed. DOW-KE backpropagates the final editing objective through the complete model, jointly optimizing the updates of all edited layers so cross-layer propagation and coupling enter every gradient step. The same principle dictates where preservation resides: embedding the preservation projection in the update parameterization, inside the computation graph, makes every gradient act on the deployed update; post-hoc constraints would reopen the gap, and the constrained search keeps edits clear of protected knowledge. In large-scale sequential editing on two datasets and three models, DOW-KE achieves the highest overall Score and neighborhood Specificity in five of six model-dataset settings among the evaluated baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。