用固定码本实现单阶段哈夫曼压缩，大幅降低大模型通信延迟。
Single-Stage Huffman Encoder for ML Compression
- 基于历史数据平均分布生成固定码本，跳过动态编码流程。
- 压缩率接近分片独立编码的0.5%以内，逼近香农理论极限。
- 适合低延迟场景如芯片间通信，无需额外传输码本开销。
训练和部署大语言模型需在多个加速器间分片数据，集体通信常受网络带宽限制。使用哈夫曼编码进行无损压缩可缓解此问题，但传统三阶段设计（实时频率分析、码本生成、传输码本）带来计算、延迟和数据开销，难以满足低延迟场景需求，如芯片间通信。本文提出单阶段哈夫曼编码器，通过采用前批次数据平均概率分布生成的固定码本，消除上述开销。通过对Gemma 2B模型的分析发现，各层与分片间的张量具有高度统计相似性。该方法实现的压缩率仅比分片级哈夫曼编码低0.5%，且距离理想香农压缩极限仅差1%，支持高效实时压缩。
原文摘要 · Abstract (English)
Training and serving Large Language Models (LLMs) require partitioning data across multiple accelerators, where collective operations are frequently bottlenecked by network bandwidth. Lossless compression using Huffman codes is an effective way to alleviate the issue, however, its three-stage design requiring on-the-fly frequency analysis, codebook generation and transmission of codebook along with data introduces computational, latency and data overheads which are prohibitive for latency-sensitive scenarios such as die-to-die communication. This paper proposes a single-stage Huffman encoder that eliminates these overheads by using fixed codebooks derived from the average probability distribution of previous data batches. Through our analysis of the Gemma 2B model, we demonstrate that tensors exhibit high statistical similarity across layers and shards. Using this approach we achieve compression within 0.5% of per-shard Huffman coding and within 1% of the ideal Shannon compressibility, enabling efficient on-the-fly compression.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。