将大模型权重压缩为统一9级量化,实现高效磁盘流式推理。
Tied Trit-Planes: Constraining PTQTP to a Uniform Nine-Level Quantizer, with a Persistent Folded Format for Disk-Streamed Mixture-of-Experts Serving
- 通过约束比例3的九级量化,将双三元平面融合为4位编码
- 在2840亿参数模型上实现零专家丢失、解码快6.7%、文件小9%
- 适合需要低延迟、高效率部署的边缘设备开发者
PTQTP将大语言模型权重矩阵分解为两个三元(三进制)平面及每组自由缩放因子。将缩放因子固定为3的比例后,分解退化为单一均匀九级量化器,即已知的平衡三进制恒等式。据我们所知,这是首次在PTQTP求解器中引入该恒等式作为约束。两个三元平面可无损折叠为一个4比特代码平面,作为持久化服务表示:磁盘字节、专家缓存字节与内核输入完全一致,均为4.0625比特/权重块,一次整数点积完成。对于这一组合(比例3九级码、CPU-SIMD内核、SSD专家流式传输、相同持久化字节),此前无先例。本方法应用于DeepSeek-V4-Flash-0731(284B-A13B MoE模型),从发布的MXFP4专家权重一次性量化,并在64GB笔记本上从SSD流式加载专家。相比4.5比特Q4_K基线,单进程每场景测试中,绑定模型在步骤0(Q4_K:4/5)和14个捕获延续步骤中的12个(11/14)匹配官方服务接口;在100项MMLU子集上得分86(基线84);解码阶段快6.7%;文件体积小9%;在小规模评估下未检测到保真度差异,所有场景差异均归因于单一实测近似持平单元。尽管绑定模型存在更高的权重重建误差与更差困惑度,反映出代理指标与真实保真度间的分离。累计主干三值化阶梯与按位锁定的aarch64/x86-64内核完整报告。全部代码、格式与评估数据开源于fucina推理栈。
原文摘要 · Abstract (English)
PTQTP decomposes LLM weight matrices into two ternary (trit) planes with two free per-group scales. Tying the scales to a fixed ratio of three collapses the decomposition into a single uniform nine-level quantizer, a known balanced-ternary identity. To our knowledge, at the time of writing, this work is the first to impose that identity as a constraint inside PTQTP's solver. The two trit planes then fold losslessly into one 4-bit code plane that we make the persistent serving representation: disk bytes, expert-cache bytes, and kernel input are the same 4.0625-bits/weight blocks, consumed in one integer dot pass. For this conjunction (ratio-3 nine-level code, CPU-SIMD kernels, SSD expert streaming, identical persistent bytes) we likewise found no precedent. We apply this to the routed experts of DeepSeek-V4-Flash-0731, a 284B-A13B mixture-of-experts model, quantizing in one shot from the released MXFP4 expert weights and streaming experts from SSD on a 64 GB laptop. Against a 4.5-bit Q4_K baseline, measured one process per fixture with an expert-lossless anchor arm as reference control, the tied model matches the official serving API on 5/5 fixtures at step 0 (Q4_K: 4/5) and 12/14 captured continuation steps (11/14), scores 86 vs. 84 on a 100-item MMLU subset, decodes 6.7% faster in decode phase, and ships 9% smaller files: no detected fidelity difference at these small evaluation sizes, and every fixture-level difference between the arms traces to a single measured near-tie cell. The tied fit nevertheless shows higher weight-reconstruction error and worse perplexity, a measured dissociation between proxy metrics and reference fidelity. A cumulative trunk-ternarization ladder and bitwise-pinned aarch64/x86-64 kernels complete the report. All code, formats, and evaluation artifacts are open source in the fucina inference stack.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。