动态平衡多模态模型编辑中的通用性与局部性,提升更新效果。
BalancEdit: Dynamically Balancing the Generality-Locality Trade-off in Multi-modal Model Editing
- 通过生成正负样本确定事实影响范围,用离散编码动态调整编辑粒度。
- 在多个数据集上实现接近零的性能损失,同时保持强编辑能力。
- 适合需要精准知识更新且不希望破坏模型整体性能的研究者使用。
大型多模态模型会随时间推移出现知识过时问题。传统微调因模型规模过大而难以应用,直接在模型内进行知识编辑成为更可行方案。然而现有方法往往忽略不同事实的影响范围差异,导致通用性与局部性之间权衡不佳。为此,本文提出多模态模型编辑中的通用性-局部性权衡概念,并构建了名为OKEDIT的新编辑数据集,专门用于评估该权衡。在此基础上,提出新方法BalancEdit,通过为每个事实生成正负样本以精确判断其影响范围,利用离散、局部化的编辑代码本将修正信息注入模型潜在空间,无需修改原始权重。实验表明,BalancEdit能有效实现通用性与局部性的动态平衡,在保持强大编辑能力的同时最小化性能下降。代码与数据集已开源。
原文摘要 · Abstract (English)
Large multi-modal models inevitably decay over time as facts update and previously learned information becomes outdated. Traditional approaches such as fine-tuning are often impractical for updating these models due to their size and complexity. Instead, direct knowledge editing within the models presents a more viable solution. Current model editing techniques, however, typically overlook the unique influence ranges of different facts, leading to compromised model performance in terms of both generality and locality. To address this issue, we introduce the concept of the generality-locality trade-off in multi-modal model editing. We develop a new model editing dataset named OKEDIT, specifically designed to effectively evaluate this trade-off. Building on this foundation, we propose \textbf{BalancEdit}, a novel method for balanced model editing that dynamically achieves an optimal balance between generality and locality. BalancEdit utilizes a unique mechanism that generates both positive and negative samples for each fact to accurately determine its influence scope and incorporates these insights into the model's latent space using a discrete, localized codebook of edits, without modifying the underlying model weights. To our knowledge, this is the first approach explicitly addressing the generality-locality trade-off in multi-modal model editing. Our comprehensive results confirm the effectiveness of BalancEdit, demonstrating minimal trade-offs while maintaining robust editing capabilities. Our code and dataset are available at https://github.com/donglgcn/BalancEdit/tree/MMOKVQA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。