用元标记实现无损压缩，让大模型推理更快更省算力
Lossless Token Sequence Compression via Meta-Tokens
- 借鉴LZ77思想，用元标记无损压缩提示词序列
- 平均压缩率27%和18%，编码计算量减少47%和33%
- 适合对语义严格保真的任务，如代码生成、语法校验
现有大语言模型提示词压缩多为有损方法，旨在保留下游任务相关语义的同时大幅缩短序列长度。本文提出一种与任务无关的无损压缩技术，类似LZ77算法，使输入令牌序列平均长度分别减少27%和18%。由于基于Transformer的模型注意力计算呈二次复杂度，该压缩对应编码计算量分别降低47%和33%。令牌序列变换可逆，确保无语义信息丢失。我们在两个需严格保持语义/语法的任务上评估，发现现有有损压缩方法表现不佳。本方法在性能上仅与未压缩输入存在微小差距，且随着模型规模扩大和计算资源增加，该差距有望完全消失。
原文摘要 · Abstract (English)
Existing work on prompt compression for Large Language Models (LLM) focuses on lossy methods that try to maximize the retention of semantic information that is relevant to downstream tasks while significantly reducing the sequence length. In this paper, we introduce a task-agnostic lossless compression technique similar to LZ77 that makes it possible to reduce the input token sequence length on average by 27\% and 18\% for the two evaluation tasks explored here. Given that we use transformer-based LLMs, this equates to 47\% and 33\% less encoding computation, respectively, due to the quadratic nature of attention. The token sequence transformation is trivial to reverse and highlights that no semantic information is lost in the process. We evaluate our proposed approach on two tasks that require strict preservation of semantics/syntax and demonstrate that existing lossy compression methods perform poorly in this setting. We find that our lossless compression technique produces only a small gap in performance compared to using the uncompressed input and posit that larger models and an expanded computing budget would likely erase the gap entirely.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。