将压缩技术扩展到低精度模型，显著降低存储与传输开销。
Lossless Compression of Neural Network Components: Weights, Checkpoints, and K/V Caches in Low-Precision Formats
- 分离指数与尾数，分别用熵编码压缩
- FP8格式压缩率最高达83%
- 适用于低精度推理场景的模型部署
随着深度学习模型规模扩大和部署普及，降低神经网络权重的存储与传输成本愈发重要。尽管此前工作如ZipNN表明，基于霍夫曼编码浮点指数的无损压缩方法能显著减小模型体积，但这些技术主要应用于FP32和BF16等高精度格式。本文将ZipNN方法扩展至更高效的低精度格式FP8和FP4。设计了一种独立压缩指数与尾数分量的方案，利用熵编码实现压缩。实验显示，该方法在BF16下可实现最高62%的压缩率，在FP8下可达83%。同时研究了大语言模型中用于推理的键值（K/V）缓存张量的可压缩性，发现其同样具有可压缩模式，可在部署时节省内存。
原文摘要 · Abstract (English)
As deep learning models grow and deployment becomes more widespread, reducing the storage and transmission costs of neural network weights has become increasingly important. While prior work such as ZipNN has shown that lossless compression methods - particularly those based on Huffman encoding floating-point exponents can significantly reduce model sizes, these techniques have primarily been applied to higher-precision formats such as FP32 and BF16. In this work, we extend the ZipNN approach to lower-precision floating-point formats, specifically FP8 and FP4, which are gaining popularity for efficient inference. We design a compression method that separates and compresses the exponent and mantissa components independently using entropy coding. Our evaluation shows compression ratios up to 62% for BF16 and 83% for FP8. We also investigate the compressibility of key-value (K/V) cache tensors used in large language models (LLMs), finding that they, too, exhibit compressible patterns, enabling memory savings during deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。