发现推理阶段缓存是决定生成结果差异的关键,精度可控制其表现。
Stage-Replay Divergence Follows the KV Cache: Fixed-Prefix Precision Controls and Bidirectional Cache Transplantation
- 通过固定前缀实验验证缓存对生成路径的决定性作用。
- FP32下无解码差异,BF16下仍有166处后缀不一致。
- 适合关注模型推理稳定性与缓存机制的研究者。
阶段重放诊断重建中间标记前缀,并将新预填充续写视为从原始解码状态延续而来。我们在Qwen2.5衍生系统中审计了整个推理阶段边界上的这一假设。一项包含200个样本的对照实验比较了保留的活跃缓存与一次性预填充相同整数标记的情况,并在两侧放置完全相同的副本。在BF16下,副本保持精确,但构造结果在166个后缀和20个正确性标签上存在差异;准确率差异仅1个百分点(配对95%置信区间[-3.5, +5.5])。固定前缀2x2实验保持全部200个标记状态不变,同时交叉构造方式与精度。BF16下的不一致仍重现,而FP32下未出现解码不一致(95% Wilson上界1.88%)。前瞻性桥梁使逐标记增量缓存与保留活跃缓存比特精确匹配12/12行;完整200项保存账本审计复现了每一条保留轨迹与对比指纹。双向移植所有48个键值层后,所有测试的分歧续写均遵循其缓存来源,在主检查点(24/24)和后续检查点的盲复现(43/43)中均成立。因此,无需保留实时状态即可实现精确标记重放。在测试状态下,阶段边界缓存是分歧轨迹的因果充分载体,而数值精度调节其行为表现。
原文摘要 · Abstract (English)
Stage-replay diagnostics reconstruct intermediate token prefixes and treat fresh-prefill continuation as continuation from the decoder state that originally reached the prefix. We audit that assumption at a whole reasoning-stage boundary in a Qwen2.5-derived system. A matched 200-item experiment compares retained live cache with one-shot prefill of identical integer tokens and places an exact replica on both sides. In BF16, replicas remain exact while the constructions differ on 166 suffixes and 20 correctness labels; the accuracy difference is only one point (paired 95% CI [-3.5, +5.5]). A fixed-prefix 2x2 holds all 200 token states constant while crossing construction and precision. The BF16 disagreements recur, whereas FP32 produces no decoded disagreement (95% Wilson upper bound 1.88%). A prospective bridge makes token-by-token incremental and retained live caches bit-exact on 12/12 rows; an all-200 saved-ledger audit reproduces every retained trajectory and comparison fingerprint. Bidirectional transplantation of all 48 key/value layers makes every tested divergent continuation follow its cache donor, both on a selected set at the primary checkpoint (24/24) and an outcome-blind replication at a later checkpoint (43/43). Exact-token replay can therefore be repeatable without preserving live-state fidelity. On the tested states, boundary K/V cache is a causally sufficient carrier of the divergent trajectory, while numerical precision moderates its behavioral expression.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。