arXiv:2510.00446cs.CLcs.SE2025-10中稿 · ASE 2025被引 45

专为代码大模型设计的上下文压缩框架,提升长代码推理效率。

LongCodeZip: Compress Long Context for Code Language Models

  • 分两阶段压缩:先选关键函数,再精筛代码块。
  • 最高压缩比达5.6倍,任务表现不下降。
  • 适合需要处理大型代码库的开发者与研究者。

随着大语言模型需在代码库中进行长上下文推理,代码生成对长上下文的需求日益突出。尽管近期进展使代码大模型能够处理长输入,但高昂的API成本和生成延迟仍是主要瓶颈。现有上下文剪枝技术(如LLMLingua)在通用文本上表现良好,却忽视了代码特有的结构与依赖关系,在编程任务中表现不佳。本文提出LongCodeZip,一种专为代码大模型设计的即插即用式压缩框架。该方法采用双阶段策略:(1) 粗粒度压缩,基于指令的条件困惑度识别并排序函数级片段,仅保留最相关函数;(2) 细粒度压缩,根据困惑度将保留函数分割为代码块,并在自适应令牌预算下选择最优子集以最大化相关性。在代码补全、摘要生成和问答等多个任务上的评估表明,LongCodeZip持续优于基线方法,最高实现5.6倍压缩比且不降低任务性能。通过有效减少上下文规模同时保留关键信息,LongCodeZip助力大模型更好适应真实世界大规模代码场景,提升代码智能应用的效率与能力。

原文摘要 · Abstract (English)

Code generation under long contexts is becoming increasingly critical as Large Language Models (LLMs) are required to reason over extensive information in the codebase. While recent advances enable code LLMs to process long inputs, high API costs and generation latency remain substantial bottlenecks. Existing context pruning techniques, such as LLMLingua, achieve promising results for general text but overlook code-specific structures and dependencies, leading to suboptimal performance in programming tasks. In this paper, we propose LongCodeZip, a novel plug-and-play code compression framework designed specifically for code LLMs. LongCodeZip employs a dual-stage strategy: (1) coarse-grained compression, which identifies and ranks function-level chunks using conditional perplexity with respect to the instruction, retaining only the most relevant functions; and (2) fine-grained compression, which segments retained functions into blocks based on perplexity and selects an optimal subset under an adaptive token budget to maximize relevance. Evaluations across multiple tasks, including code completion, summarization, and question answering, show that LongCodeZip consistently outperforms baseline methods, achieving up to a 5.6x compression ratio without degrading task performance. By effectively reducing context size while preserving essential information, LongCodeZip enables LLMs to better scale to real-world, large-scale code scenarios, advancing the efficiency and capability of code intelligence applications.

代码生成上下文压缩大模型优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。