提出检测压缩令牌溢出的方法，提升长文本生成的准确性。
Detecting Overflow in Compressed Token Representations for Retrieval-Augmented Generation
- 通过查询感知探测器识别压缩信息丢失
- 在多个数据集上平均达到0.72 AUC-ROC
- 适合资源受限环境下大模型优化
高效处理长上下文仍是当前大语言模型在资源受限环境下的关键挑战。软压缩架构通过将长令牌序列替换为少量学习得到的压缩令牌，有望扩展有效上下文长度。然而，压缩能力的极限——即压缩开始抹去任务相关内容的临界点——仍缺乏深入研究。本文定义了‘令牌溢出’为压缩表示不再包含足够信息回答给定查询的状态，并提出一种表征与检测方法。在xRAG软压缩设置中，我们发现仅依赖查询无关的饱和统计量能可靠区分压缩与未压缩表示，但溢出检测能力有限。而对查询和上下文的xRAG表示施加轻量级探测分类器，在HotpotQA、SQuADv2和TriviaQA数据集上平均实现0.72 AUC-ROC，表明引入查询信息可显著提升检测性能。该成果推动从查询无关诊断迈向查询感知检测，实现低成本预LLM门控，缓解压缩导致的错误。
原文摘要 · Abstract (English)
Efficient long-context processing remains a crucial challenge for contemporary large language models (LLMs), especially in resource-constrained environments. Soft compression architectures promise to extend effective context length by replacing long token sequences with smaller sets of learned compressed tokens. Yet, the limits of compressibility -- and when compression begins to erase task-relevant content -- remain underexplored. In this paper, we define token overflow as a regime in which compressed representations no longer contain sufficient information to answer a given query, and propose a methodology to characterize and detect it. In the xRAG soft-compression setting, we find that query-agnostic saturation statistics reliably separate compressed from uncompressed token representations, providing a practical tool for identifying compressed tokens but showing limited overflow detection capability. Lightweight probing classifiers over both query and context xRAG representations detect overflow with 0.72 AUC-ROC on average on HotpotQA, SQuADv2, and TriviaQA datasets, demonstrating that incorporating query information improves detection performance. These results advance from query-independent diagnostics to query-aware detectors, enabling low-cost pre-LLM gating to mitigate compression-induced errors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。