arXiv:2606.12203cs.CL2026-06被引 1

用可变粒度软令牌压缩技能,降低大模型推理开销。

Adaptive Multi-Resolution Procedural Knowledge Compression for Large Language Models

论文配图:Adaptive Multi-Resolution Procedural Knowledge Compression for Large Language Models
图 1 · 摘自论文原文
  • 根据技能复杂度动态生成不同数量的软令牌
  • 压缩后仅保留原长度30%~60%,任务效果更优
  • 适合频繁更新的社区技能,支持离线压缩

大型语言模型(LLMs)广泛用于处理复杂任务并执行自主工作流。近年来,可复用的自然语言技能成为向LLM应用注入过程知识的流行范式。由于常用技能频繁调用,将完整文本置于每个上下文中会显著增加预填充成本和延迟。尽管文本压缩技术有潜力解决此问题,但现有方法多针对文档中的事实知识压缩,不适用于过程知识。本文提出SKIM(SKIll coMpression),一种自适应多分辨率软令牌压缩框架,能根据技能复杂度动态生成软令牌数量,既提升推理效率,又保持技能使用效果。实验表明,SKIM可将技能压缩至原始长度的30%至60%,且性能优于现有方法。代码已开源:https://github.com/bebr2/SKIM。

原文摘要 · Abstract (English)

Large language models (LLMs) are widely used to tackle complex tasks with autonomous workflows. Recently, reusable natural language skills have emerged as a popular paradigm to inject procedural knowledge into LLM applications. Since popular skills are often invoked repeatedly, placing their full text in every context significantly increases prefill cost and latency. While text compression techniques have the potential to solve this problem, most existing methods are designed to compress factual knowledge in documents instead of procedural knowledge, making them insufficient for skill compression. In this paper, we argue that an effective skill compression method should: 1) preserve logical dependencies among workflows and tool protocols, 2) enable lightweight, offline compression for frequently updated community skills, and 3) be adaptable to varying complexities across skills. To address this, we present SKIM (SKIll coMpression), an adaptive multi-resolution soft token compression framework for procedural skills. Depending on the complexity of each skill, SKIM creates different numbers of soft tokens that not only improve the efficiency of LLM inference, but also preserve the effectiveness of skill usage. Experiments indicate that SKIM compresses skills to 30 to 60 percent of their original token length while preserving task performance better than existing compression methods.We have released our code at https://github.com/bebr2/SKIM .

大模型技能压缩软令牌推理优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。