通过任务引导实现跨查询复用的缓存压缩，提升长文本推理效率。
TaskPress: Query-Agnostic KV Cache Compression via Task-Guided Pruning
- 基于任务引导构建可复用的缓存表示，摆脱对单个查询的依赖。
- 利用量化尺度因子识别关键特征，零成本探测重要令牌。
- 在多种长文本任务中实现高效、通用的缓存压缩，适合大模型推理优化者。
大语言模型进行长上下文推理时，受键值缓存随序列长度线性增长的限制。尽管剪枝可缓解该问题，但现有方法依赖于查询特定的令牌重要性评估，无法在未见查询间复用。为此，我们提出TaskPress，一种任务引导的、查询无关的键值缓存剔除框架。与针对单个查询优化缓存不同，TaskPress基于高层任务指导构建可复用的记忆表征。该指导在预填充阶段充当元查询，过滤无关令牌，为后续查询提供支持。此外，TaskPress利用量化尺度因子作为零成本信号，检测具有影响力的表示异常值，从而高效代理令牌重要性。在多个包含长上下文输入的任务上进行的实验表明，TaskPress能有效生成紧凑且跨多样化查询可复用的缓存。
原文摘要 · Abstract (English)
Long-context inference with large language models is constrained by the linear growth of the key-value cache to sequence length. While pruning offers mitigation, prevailing methods determine query-specific token importance that cannot be reused across unseen queries. In contrast, we introduce TaskPress, a framework for task-guided, query-agnostic KV cache eviction. Instead of optimizing the cache for a single query, TaskPress constructs a reusable memory representation conditioned on a high-level task guide. The guide functions as a meta-query during prefill to filter irrelevant tokens before downstream queries are issued. In addition, TaskPress leverages quantization scale factors as a zero-cost signal for detecting influential representation outliers, providing an efficient proxy for token importance. Experiments on conducted on various tasks with long context input demonstrate that TaskPress efficiently creates a compact, reusable cache across diverse queries.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。