arXiv:2607.17486cs.PFcs.AI2026-07

用词频字典树实现长文本压缩,避免主题丢失。

SALT: Salience-Aware Lexical Trie for Long-Context Compression

论文配图:SALT: Salience-Aware Lexical Trie for Long-Context Compression
图 1 · 摘自论文原文
  • 构建按句子频率排序的词频字典树,捕捉文档主题结构
  • 多锚点检索支持跨对话保持主题覆盖,节省计算与内存
  • 不依赖模型,可与缓存优化方法兼容,适合长对话场景

随着大语言模型处理更长提示,推理系统中的计算和键值缓存(KV-cache)内存成本成为主要瓶颈。现有输入级提示压缩方法通过标量相关性评分对句子排序,将文档视为无结构的词句池,导致在预算紧张时出现主题坍塌——主导主题占用全部资源,忽略次要但任务相关的主题。为保留主题覆盖率,需将预算分配给重复出现的主题,而非孤立评估句子。为此,我们提出SALT:一种模型无关的抽取式压缩框架,将每句关键词组织成按句子频率(SF)排序的字典树,作为文档主题结构的轻量级、可复用代理。该字典树结构平滑内存分配,防止主导主题垄断预算。多锚点检索可在任意深度激活由查询关键词标记的字典节点,且字典树可跨对话轮次持续使用,无需重新编码文档。SALT在减少长上下文提示的预填充计算与内存开销的同时,仍可与针对解码延迟和内存优化的KV-cache方法组合使用。

原文摘要 · Abstract (English)

As large language models (LLMs) process increasingly longer prompts, computation and KV-cache memory costs have emerged as major bottlenecks in inference systems. Existing input-level prompt compression methods address this, but rank each sentence by a scalar relevance score, treating the document as an unstructured pool of words and sentences. Under tight budgets, this causes theme collapse, where the dominant theme(s) of a document consumes the budget, discarding less-frequent yet task-relevant themes. Preserving thematic coverage instead requires allocating the budget across recurring themes rather than scoring sentences in isolation. To this end, we propose SALT, a model-agnostic extractive framework that organizes per-sentence keywords into a trie ordered by sentence frequency (SF), a lightweight, reusable proxy for document thematic structure. This trie-based organization smooths memory allocation and prevents dominant themes from monopolizing the budget. Multi-anchor retrieval activates trie nodes labeled by query keywords at any depth, and the trie persists across dialogue turns, supporting multi-turn use without re-encoding the document. By preserving document themes, SALT reduces the prefill computation and memory cost of long-context prompts while remaining composable with KV-cache methods that target decoding-time latency and memory.

长文本压缩主题保持字典树多轮对话

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。