用压缩向量提升代码补全效率，显著降低延迟。
CoRoVA: Compressed Representations for Vector-Augmented Code Completion
- 将检索上下文压缩为少量语义丰富的向量
- 相比原始检索增强方法，首令牌生成时间降低20%-38%
- 仅需微小额外开销，适合交互式开发场景
检索增强生成已成为提升代码补全性能的有效方法，尤其在需要仓库级上下文时。然而，引入额外检索内容会显著增加序列长度，提高预填充成本，并恶化首次令牌生成时间（TTFT），从而影响推理速度——这对集成开发环境（IDE）等交互场景构成关键限制。本文提出 CoRoVA 框架，将上下文压缩为紧凑且语义丰富的表示，保持对代码大模型的可读性。该方法仅需训练一个小型投影模块，引入可忽略的额外延迟，却能显著提升代码大模型的预测质量。实验表明，与未压缩的 RAG 相比，CoRoVA 在补全任务中实现 20%-38% 的 TTFT 下降。
原文摘要 · Abstract (English)
Retrieval-augmented generation has emerged as one of the most effective approaches for code completion enhancement, especially when repository-level context is important. However, adding this extra retrieved context significantly increases sequence length, raises prefill cost, and degrades time-to-first-token (TTFT), which slows down inference -- a critical limitation for interactive settings such as IDEs. In this work, we introduce CoRoVA, a framework that compresses context into compact, semantically rich representations that remain interpretable to code LLMs. This improves generation quality while reducing prompt augmentation to only a few compressed single-token vectors. Our approach requires training only a small projector module and introduces negligible additional latency, yet it significantly improves the prediction quality of code LLMs. Our experiments show that CoRoVA enables a 20-38\% reduction in TTFT on completion tasks compared to uncompressed RAG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。