arXiv:2607.28699cs.ARcs.AI2026-07被引 2

为键值缓存量化提供实时风险监测与防护,确保推理质量不下降。

WitCert: Sound Runtime Risk Observability and Gating for KV-Cache Quantization

论文配图:WitCert: Sound Runtime Risk Observability and Gating for KV-Cache Quantization
图 1 · 摘自论文原文
  • 提出分层头步级的压缩误差上界测量方法,支持任意量化器。
  • 实测在硬任务上使FP8精度从22.8提升至79.7,误差控制在[+0.0,+0.8]。
  • 适合关注推理稳定性与内存效率的系统开发者和部署工程师。

当前KV缓存量化依赖离线平均指标,无法感知实时请求的质量损伤。本文提出可证明可靠的运行时度量——"针对KV量化的DTrace":对每一层、头、步骤提供精确与压缩注意力间总变差的上界。该度量分为两层:一是适用于任何缓存保留黑盒量化器及任意查询的确定性带范数见证界(自适应安全,基于柯西-施瓦茨与RoPE带单位性);二是针对受控减法抖动INT8量化器,在请求级失败预算下的更紧概率证书(针对非自适应查询;核心定理经Lean 4机器验证)。三个成果:可观测性——度量通过环境防护补丁接入SGLang,任何注册为张量函数的方案均可在真实服务中测量;修复性——基于度量的风险排序与认证决策,实测恢复基准尺度质量下限,如在困难的RULER任务上,原始FP8从22.8提升至79.7,与未压缩相比差异控制在[+0.0,+0.8]的配对检验范围内;分析发现:激进方案存活依赖跨层误差抵消,而非单步保真度——28层扫描中无一单独层级导致损失(0/28),且认证后的INT8缓存可在相同内存下处理1.88倍更多KV token。所有代码、防护机制及Lean开发均开源于https://github.com/metask-ai/witcert-kv-certificates;每个数字仅需一条命令即可从发布资产重生成。

原文摘要 · Abstract (English)

KV-cache quantization is validated today by offline benchmark averages; a deployed system cannot tell whether compression is damaging the request it is serving right now. We give it a provably sound runtime meter -- a "DTrace for KV quantization": a per-(layer, head, step) upper bound on the total variation between exact and compressed attention. The meter has two tiers: a deterministic band-norm-witness bound, sound for any cache-preserving black-box quantizer and for any query (adaptive-safe, worst-case Cauchy--Schwarz plus RoPE band-unitarity), and a tighter probabilistic certificate for a controlled subtractively-dithered INT8 quantizer under an explicit request-level failure budget (stated for non-adaptive queries; core theorems machine-checked in Lean 4). Three results. Observability: the meter enters SGLang through an env-guarded patch, and any scheme registered as one tensor function is measured in live serving. Repair: meter-driven gating -- risk-ranked where the witness is saturated, certified where it is informative -- empirically restores the quality floor at benchmark scale, e.g. raw-cast fp8 from 22.8 back to 79.7 on hard RULER tasks with the difference from uncompressed bounded at $[+0.0,+0.8]$ by a paired test. Analysis: aggressive schemes survive on cross-layer error cancellation, not per-step fidelity -- in a 28-layer sweep, no single layer's pollution alone loses anything (0/28) -- and the certified int8 cache serves $1.88\times$ more KV tokens at the same memory in SGLang. All artifacts, guards, and the Lean development are released at https://github.com/metask-ai/witcert-kv-certificates; every number regenerates from the shipped artifacts by one command.

量化推理优化风险监测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。