让编程助手每次对话节省95%上下文，还能准确保留关键代码片段。
Paritok-4B: Intent-Conditioned Context Compression for Coding Agents
- 只选不改：从原始文本中直接提取代码片段，96%的标识符和路径原样保留。
- 按任务精选：根据当前目标选择保留哪行代码，比随机删减更精准。
- 轻量高效：264MB小模型，单张显卡就能运行，省钱又省资源。
编程代理每轮都会向前沿大模型发送大量文件读取内容和工具输出，导致上下文占用了绝大部分令牌开销。通用提示压缩器训练于普通文本，对代码处理效果差：会改写标识符、丢失代理需要修改的精确片段。我们提出 Paritok-4B，一个基于两个承诺构建的 40 亿参数 LoRA 压缩器。它是提取式的：不重写，而是直接选取原文中的片段，96.0% 的标识符、路径和数字在输出中已存在于输入，在独立测试集 SWE-bench Lite 上保持 96.2%。它是意图感知的：给定代理当前任务，它主要在保留段内选择哪些行应被留下（保留行比删除行更相关，平均高出 +0.067，95% 置信区间 [+0.056, +0.078]），而非决定保留多少内容。我们通过 67,074 条真实 OpenHands 轨迹，将 gpt-4.1-mini 教师模型蒸馏为 40,606 个验证样本，并微调 Qwen3-4B。在全部 300 个 SWE-bench Lite 实例上，该模型将上下文压缩至原始大小的 25.7%，比 gpt-4.1-mini（50.2%）难压缩 2.0 倍，比 gpt-5（61.9%）难压缩 2.4 倍，同时保留了 86.5% 的单次求解质量。使用真实代理生成的 -n 行号输入时，压缩率为 27.8%，保留率更高达 89.3%；配对测试显示，30 个实例仅在未压缩时成功解决，17 个仅在压缩后成功，麦纳玛尔检验得 p=0.079，表明在该样本量下，将上下文压缩至四分之一左右不会显著降低求解成功率。该模型为 264 MB 适配器，可在单张 24 GB 显卡上自托管，无每令牌压缩费用。按列表价格计算，以 gpt-5 作为压缩器净成本为负，花费高于其节省的下游令牌数。权重、数据与评估脚本已开源（Apache 2.0）。
原文摘要 · Abstract (English)
Coding agents re-send large file reads and tool outputs to a frontier LLM every turn, and this context dominates their token bill. General-purpose prompt compressors are trained on prose and suit code poorly: they paraphrase identifiers and drop the exact spans an agent needs to edit. We present Paritok-4B, a 4B LoRA compressor for coding-agent trajectories built on two commitments. It is extractive: it selects spans rather than rewriting them, and 96.0% of the identifiers, paths, and numbers it emits already appear in its input, holding at 96.2% on held-out SWE-bench Lite output. It is intent-conditioned: told the agent's current task, it acts chiefly inside a retained segment, selecting which lines survive (retained lines are +0.067 more intent-relevant than removed ones, paired 95% CI [+0.056, +0.078]) rather than changing how much is retained. We distil a gpt-4.1-mini teacher over 67,074 real OpenHands trajectories into 40,606 validated examples and fine-tune Qwen3-4B. On all 300 SWE-bench Lite instances, Paritok-4B compresses agent context to 25.7% of its size, 2.0x harder than a gpt-4.1-mini compressor (50.2%) and 2.4x harder than gpt-5 (61.9%), while retaining 86.5% of uncompressed single-shot solve quality. Fed the cat -n line-numbered input real agents produce, it compresses slightly less (27.8%) and retains more (89.3%); there the paired test is informative, with 30 instances solved only uncompressed and 17 only compressed, an exact McNemar p=0.079, so at this sample size compressing context to roughly a quarter of its size does not significantly reduce the solve rate. The model is a 264 MB adapter that self-hosts on one 24 GB GPU with no per-token compressor fee, which at list prices decides the economics: gpt-5 as a compressor is net-negative, costing more than the downstream tokens it saves. Weights, data, and evaluation scripts are open (Apache 2.0).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。