用大模型预测文本字符，再压缩预测结果，提升无损压缩效率。
AlphaZip: Neural Network-Enhanced Lossless Text Compression
- 用Transformer预测下一个字符的可能候选排名
- 在WikiText-103上比传统算法压缩率高12.7%
- 适合对压缩率敏感的文本存储与传输场景
数据压缩持续演进，传统信息论方法广泛用于文本、图像和视频压缩。近年来，生成式AI在预测性压缩技术中受到越来越多关注。本文提出一种基于大语言模型（LLM）的无损文本压缩方法，包含两个关键步骤：首先，使用密集神经网络架构（如Transformer块）进行预测；其次，利用自适应霍夫曼编码、LZ77或Gzip等标准压缩算法对预测出的字符排名进行压缩。在多种基准测试中，该方法与传统信息论基线相比表现出更优性能，在WikiText-103数据集上压缩率提升达12.7%。
原文摘要 · Abstract (English)
Data compression continues to evolve, with traditional information theory methods being widely used for compressing text, images, and videos. Recently, there has been growing interest in leveraging Generative AI for predictive compression techniques. This paper introduces a lossless text compression approach using a Large Language Model (LLM). The method involves two key steps: first, prediction using a dense neural network architecture, such as a transformer block; second, compressing the predicted ranks with standard compression algorithms like Adaptive Huffman, LZ77, or Gzip. Extensive analysis and benchmarking against conventional information-theoretic baselines demonstrate that neural compression offers improved performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。