为异构注意力记忆设计运行时可观测性系统,精准追踪压缩下的内存故障。
Runtime Observability for Heterogeneous Attention Memory
- 构建三操作符合约覆盖四类记忆,实现请求级风险量化
- 在1240万次读取中零违规,自动分级认证每项结论可靠性
- 适用于部署中的大模型,尤其适合高压缩场景的故障定位
现代模型不再使用简单的键值缓存:潜在缓存、学习型稀疏选择器和循环状态以不同形式承载模型记忆,且在压缩下各有不同的失效模式。本文提出一种涵盖四类记忆的运行时可观测性契约,包含三个操作符,并在五个架构族的六种模型配置上实现。将各阶段边界组合成可执行的请求级风险账本。契约自带误差度量作为类型——仅当度量匹配时才允许组合,此检查曾否决我们首个组合链;修复后的链通过两个经证明的桥梁传递度量,无法形式化验证的部分则改用实测,将组合层级降为经验级:每个断言被标记为完全认证、部分认证或经验级,组合继承最弱层级,由机器判定。在八路并发、每请求预算约束及失败关闭身份归属下,回放1240万次读取,账本精确量化当前见证下的诚实权衡,风险预算零违反。一个融合的始终开启探针,在服务噪声底限内通过CUDA图观测声明的一层子集。应用于部署的DeepSeek-V4堆栈与压缩键值原型,同一机制将一次静默损坏精确定位至特定结构边界——在无驱逐、身份隔离的场景下精确,所有观察到的故障均发生在驱逐或槽复用场景中——通过机器裁定的区分行动完成,过程中拒绝了我们自身两个混淆推断。所有工具、保护机制和精益开发过程已开源于https://github.com/metask-ai/witprobe-attention-memory;本文所有数据均可通过一条命令从发布代码重生成。
原文摘要 · Abstract (English)
Modern models no longer keep a plain KV cache: latent caches, learned sparse selectors and recurrent states each carry the model's memory in a different form, and each fails differently under compression. We give a runtime observability contract that covers all four memory classes with three operators, instantiate it on six model configurations across five architecture families, and compose the per-stage bounds into an executable request-level risk ledger. Contracts carry their error metric as a type -- composition is only defined when metrics match, and this check rejected our own first composed chain; the repaired chain crosses metrics through two proved bridges, and whatever no formal system can certify is measured instead, dropping the composed tier to empirical automatically: every claim is certified, partially certified, or empirical, composition inherits the weakest tier, and the tier is decided by the machine. Replayed over $12.4$M entry reads and run under eight-way concurrency with per-request budgets and fail-closed identity attribution, the ledger quantifies the honest trade-off on today's witness and holds its risk budget with zero violations. A fused always-on probe observes a declared one-layer subset under CUDA graphs inside the serving noise floor. Applied to a served DeepSeek-V4 stack with a packed compressed-KV prototype, the same machinery localizes a silent corruption to a precise structural boundary -- exact in the eviction-free, identity-isolated regime, with every observed failure in an eviction or slot-reuse regime -- through a machine-adjudicated discrimination campaign whose calculus rejected two of our own confounded inferences along the way. All artifacts, guards, and the Lean development are released at https://github.com/metask-ai/witprobe-attention-memory; every number in this paper regenerates from the shipped artifacts by one command.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。