让小模型瞬间变强又省钱,靠的是精确复制知识缓存。
Smarter and Cheaper at Once: Byte-Exact KV-Cache Grafting Turns a Frozen Small Model into a Verified-Knowledge Flywheel
- 通过精准嫁接知识缓存,无需修改模型权重即可提升性能。
- 在AIME 2025上准确率从80%提升至93.3%,节省超6000倍推理资源。
- 知识可跨设备无损复用,适合需要高效推理的部署场景。
我们提出一种方法,使冻结的小型语言模型在不更改任何权重的前提下,同时提升能力并大幅降低计算成本。验证过的知识以字节精确的键值(KV)状态形式一次性存储,并通过嫁接技术在新推理上下文中完全还原。该还原过程为位级精确:在固定确定性配置下,嫁接后的输出与全新计算结果完全一致(SHA-256哈希相等),零KL散度,五十样本中100%匹配最大概率词。我们证明,位置自洽嫁接是使用浮点旋转编码模型的唯一数值精确操作点,并在两个模型规模(12B、31B)和两个GPU目标上验证了字节精确性,其中一个通过预注册回放验证。在AIME 2025测试中,冻结的Gemma-4-12B模型准确率从80.0%提升至93.3%,超过其自身77.5%和31B版本的89.2%基准表现。在重复案例中,原本在401,026个标记预算内无法解决的8个问题,仅用61个解码标记即从缓存中恢复答案,减少约6,574倍标记数和约8,700倍能耗;能力验证基于独立迁移测试(31B上7/7成功)。相同字节精确存储将可用上下文长度从32,768扩展至2,854,766,且不增加加速器内存开销,可在同架构机器间无缝迁移。系统描述基于行为层面;引擎为专有实现,所有报告数据均附带输入输出哈希,可独立复核评分。
原文摘要 · Abstract (English)
We report a way to make a frozen small language model both more capable and dramatically cheaper at once, without changing any weights. Verified knowledge is deposited once as a byte-exact key-value (KV) state artifact and later restored, by graft, into a fresh inference context. The restore is bit-exact: under a pinned deterministic configuration, the grafted logits are byte-for-byte identical to a fresh computation (SHA-256 equality), with zero KL divergence and 100% argmax agreement over fifty samples. We show that own-position graft is the unique numerically exact operating point on a model with floating-point rotary encoding, and we verify byte-exactness on two model scales (12B, 31B) and two GPU targets, one through a pre-registered replay. On AIME 2025, a frozen Gemma-4-12B moves from 80.0% to 93.3% once a verified solution library is grafted, above its own 77.5% and its 31B sibling's 89.2% published anchors. On the recurring case, eight problems the base model never solves within a 401,026-token budget are answered from cached verified solutions in 61 total decode tokens, a factor of 6,574 fewer tokens and about 8,700x less energy; the capability claim proper rests on held-out transfer (7 of 7 at 31B). The same byte-exact store widens usable context from 32,768 to 2,854,766 tokens at zero extra accelerator memory, and moves byte-identical between machines of the same architecture. We describe the system at the behavior level; the engine is proprietary, and every reported number is backed by committed input and output hashes so the scoring can be re-checked without it.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。