arXiv:2608.20873cs.LG2026-08

让大模型在不改原文件的前提下,动态更新知识并保持量化精度不变。

In-Cell Learning: Language Models That Update Their Own Weights in Sequence Without Changing the File They Ship

论文配图:In-Cell Learning: Language Models That Update Their Own Weights in Sequence Without Changing the File They Ship
图 1 · 摘自论文原文
  • 在量化权重的固定单元内更新参数,保持原始文件比特级一致
  • 在多个模型上实现99%知识注入,88%问答准确率提升
  • 适合需要持续更新且严格版本管理的部署场景

4比特量化权重定义了一个四舍五入单元而非单一高精度值。我们提出‘单元内学习’范式,仅在这些单元内写入新知识,使得重新量化服务权重时能精确复现已发布的整数代码和缩放系数。CellFill通过在冻结的量化单元中设置有限可训练位置,将更新以独立、可减法撤销的文件形式发布。在从1.7B到32B参数的Qwen3与Gemma系列公开的NF4和W4A16版本中,CellFill在真实事实语料库上完成了83%-99%的知识写入,并在每个约束权重上返回存储代码。注入的事实可泛化至改写与组合任务,使模型解答了78%-88%原本遗漏的PopQA问题。序列实验表明,重演机制维持早期知识,但可用空间与新任务可塑性随更新递减。整合操作重新量化学习后的权重,生成明确的重大版本,恢复空间但带来可测量的能力损失。六次写入-重演-整合循环中,两个80亿参数运行保留至少92.8%初始学习能力,69亿个约束权重每轮均无代码违规。这些结果定义了一种版本管理协议:小版本更新保持发布量化产物比特级一致,大版本更新则显式、可度量、可验证。

原文摘要 · Abstract (English)

A 4-bit quantized weight specifies a rounding cell rather than a single full-precision value. We introduce in-cell learning, a paradigm for writing new knowledge only within these cells, so that re-quantizing the served weights reproduces the released integer codes and scales exactly. CellFill implements this idea with bounded trainable positions inside frozen quantization cells and ships the update as a separate, subtractively revocable file. Across published NF4 and W4A16 releases of Qwen3 and Gemma from 1.7B to 32B parameters, CellFill writes 83-99% of a real-fact corpus while returning the stored code on every constrained weight. The injected facts generalize to paraphrases and composition, and answer 78-88% of selected PopQA questions that the released model misses. Sequential experiments show that rehearsal preserves earlier knowledge, whereas available room and new-task plasticity decline across updates. Consolidation re-quantizes the learned weights to produce a declared major version, restoring room at a measured capability cost. A six-task write-rehearse-consolidate cycle retains at least 92.8% of first learning in two 8B runs and records zero code violations over 6.9 billion constrained weights at every fold. These results define a version-management protocol in which minor updates preserve the released quantized artifact bitwise and major updates are explicit, measurable, and verifiable.

模型更新量化知识注入版本管理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。