提出HORSE方法，实现大模型精准批量编辑且更稳定
Hierarchical Orthogonal Residual Spread for Precise Massive Editing in Large Language Models
- 通过分层正交残差扩散，减少梯度噪声
- 在多个大模型上实现精准批量编辑，效果优于主流方法
- 适合需要大规模安全修正的场景，如内容过滤
大语言模型（LLMs）在多个领域表现优异，但存在严重安全问题。模型编辑已成为缓解此类问题的有效手段。现有方法通常优化一个混合新旧知识的信息矩阵，虽有效但计算成本高且易产生冲突。本文提出分层正交残差扩散（HORSE），从不同角度减少梯度噪声，实现更稳定的编辑。通过与多种主流方法的理论对比及在两个数据集上对多个大模型的广泛实验，验证了HORSE在多样场景下保持精确大规模编辑的能力。代码已开源。
原文摘要 · Abstract (English)
Large language models (LLMs) exhibit exceptional performance across various domains, yet they face critical safety concerns. Model editing has emerged as an effective approach to mitigate these issues. Existing model editing methods often focus on optimizing an information matrix that blends new and old knowledge. While effective, these approaches can be computationally expensive and may cause conflicts. In contrast, we shift our attention to Hierarchical Orthogonal Residual SprEad of the information matrix, which reduces noisy gradients and enables more stable edits from a different perspective. We demonstrate the effectiveness of our method HORSE through a clear theoretical comparison with several popular methods and extensive experiments conducted on two datasets across multiple LLMs. The results show that HORSE maintains precise massive editing across diverse scenarios. The code is available at https://github.com/XiaojieGu/HORSE
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。