arXiv:2602.18628cs.LGcs.AI2026-02

让模型参数变成可无限扩展的函数,解决知识遗忘问题。

Non-Interfering Weight Fields: Treating Model Parameters as a Continuously Extensible Function

  • 将固定权重改为连续函数,按需生成参数配置。
  • 在Mistral-7B上实现零遗忘,新任务性能接近原模型。
  • 支持能力版本管理,可组合、回滚,无需重新训练。

大型语言模型将所有知识存储于单一固定权重向量中。学习新能力需修改原有权重,必然导致已有知识退化,即灾难性遗忘,这一根本问题长期缺乏有效解决方案。现有方法将权重视为不可变实体,依赖正则化、重放缓冲区或独立适配模块来保护,但均无结构性保障。本文提出非干扰权重场(NIWF),以可学习函数替代固定权重,从连续能力坐标空间动态生成权重配置。完成任务训练后,通过锚点快照锁定该区域输出,确保后续训练不破坏已学内容。在Mistral-7B上验证,顺序指令遵循与代码生成任务中实现已承诺任务零遗忘,新任务困惑度保持竞争力。该框架引入类软件版本控制的神经智能管理机制,使能力可提交、扩展、组合与回滚,无需重新训练。

原文摘要 · Abstract (English)

Large language models store all learned knowledge in a single, fixed weight vector. Teaching a model new capabilities requires modifying those same weights, inevitably degrading previously acquired knowledge. This fundamental limitation, known as catastrophic forgetting, has resisted principled solutions for decades. Existing approaches treat weights as immutable artifacts that must be protected through techniques like regularization heuristics, replay buffers, or isolated adapter modules. The problem is none of these provide a structural guarantee against forgetting. In this work, we propose Non-Interfering Weight Fields (NIWF), a framework that replaces the fixed weight paradigm with a learned function that generates weight configurations on demand from a continuous capability coordinate space. After training on a task, we commit the occupied coordinate region by snapshotting the fields outputs on anchor points to enforce a functional lock during all future training. We validate NIWF on sequential instructionfollowing and code generation tasks using Mistral-7B, demonstrating zero forgetting on committed tasks with competitive perplexity on new tasks. The framework introduces the notion of software-like versioning for neural network intelligence, where capabilities can be committed, extended, composed, and rolled back without retraining.

权重函数知识保存版本管理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。