用大模型压缩数据，实现在enwik9上18%的先进压缩率。
Revisiting Data Compression with Language Modeling
- 用大模型作为压缩器，不需额外训练即达新最优。
- 在enwik9数据集上实现约18%的调整后压缩率。
- 对非英语、代码、字节流等数据仍具竞争力。
本报告探讨了大语言模型（LLM）在数据压缩任务中的潜力。已有研究证明LLM不仅能压缩文本，还能处理多种多模态数据。尽管性能优异，但其替代现有压缩算法仍面临实际挑战。本文探索不同方法以降低调整后的压缩率。相比先前工作，在enwik9数据集上未进行额外训练即实现了约18%的最新状态（SOTA）调整压缩率。此外，我们研究了LLM在非英语数据、代码数据和字节流序列上的压缩表现。结果表明，尽管LLM在以文本为主的数据域中表现卓越，但在正确配置下，其在非自然文本序列上的压缩能力依然具有竞争力。
原文摘要 · Abstract (English)
In this report, we investigate the potential use of large language models (LLM's) in the task of data compression. Previous works have demonstrated promising results in applying LLM's towards compressing not only text, but also a wide range of multi-modal data. Despite the favorable performance achieved, there still remains several practical questions that pose a challenge towards replacing existing data compression algorithms with LLM's. In this work, we explore different methods to achieve a lower adjusted compression rate using LLM's as data compressors. In comparison to previous works, we were able to achieve a new state-of-the-art (SOTA) adjusted compression rate of around $18\%$ on the enwik9 dataset without additional model training. Furthermore, we explore the use of LLM's in compressing non-English data, code data, byte stream sequences. We show that while LLM's excel in compressing data in text-dominant domains, their ability in compressing non-natural text sequences still remain competitive if configured in the right way.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。