提出一种抗预测偏差的无损压缩算法，突破传统模型需完全一致的限制。
A Model-Driven Lossless Compression Algorithm Resistant to Mismatch
- 基于下一词预测构建抗结构化偏差的压缩框架
- 在可认证的预测差异范围内实现可靠压缩，优于主流方法
- 适合使用复杂神经网络模型的场景，如大语言模型
由于下一符号预测与压缩之间存在根本联系，现代预测模型（如大语言模型）可与熵编码结合，实现超越传统压缩算法的压缩率。然而，该方法依赖编码器与解码器生成相同输出分布的假设，任何微小偏差都可能导致解码失败。这一假设在复杂模型（尤其是神经网络）中常不成立，称为非确定性现象。本文提出一种基于下一词预测的新型压缩算法，能抵御任意但结构化的预测偏差。我们通过形式化偏差认证证明了该方案的正确性，刻画其理论性能，并在真实数据集上实验验证。结果表明，在可认证的偏差范围内系统稳定运行，压缩率高于常用压缩方法。
原文摘要 · Abstract (English)
Due to the fundamental connection between next-symbol prediction and compression, modern predictive models, such as large language models (LLMs), can be combined with entropy coding to achieve compression rates that surpass those of standard compression algorithms. However, this approach relies on the assumption that the predictive model produces identical output distributions at both the encoder and decoder, since even small mismatches can cause the decoding to fail. This assumption often fails with complex predictive models, particularly those based on neural networks, a phenomenon referred to as non-determinism. In this work, we propose a new compression algorithm based on next-token prediction that is robust to arbitrarily large, but structured, prediction mismatches. We prove the correctness of the proposed scheme under a formal mismatch certification, characterize its theoretical performance, and validate it experimentally on real datasets. Our results demonstrate reliable operation within the certified mismatch regime while achieving compression ratios that exceed those of commonly used compression methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。