arXiv:2606.18023cs.LGcs.AI2026-06被引 1

通过优化循环次数提升代码生成效率,发现两轮循环效果最佳。

LoopCoder-v2: Only Loop Once for Efficient Test-Time Computation Scaling

论文配图:LoopCoder-v2: Only Loop Once for Efficient Test-Time Computation Scaling
图 1 · 摘自论文原文
  • 采用并行循环结构,用位置偏移和共享键值缓存降低延迟
  • 两轮循环使SWE-bench得分从43.0升至64.4,三轮以上反而下降
  • 揭示循环次数存在收益-成本权衡,适合高效推理场景

并行循环变压器(PLT)通过跨循环位置偏移(CLP)和共享键值缓存滑动窗口注意力,缓解了传统循环结构带来的延迟与内存开销。本文以收益-成本视角研究循环次数选择,训练了从零开始的7B PLT代码模型家族,基于18T token数据进行指令微调与评估。实验表明,两轮循环在代码生成、代码推理、智能软件工程和工具使用等任务上显著优于无循环基线,其中SWE-bench Verified得分由43.0提升至64.4,Multi-SWE由14.0升至31.0。而三轮及以上循环则出现性能退化,呈现强非单调性。诊断显示第二轮循环带来主要表征优化,后续循环产生边际递减、振荡更新且表征多样性下降。由于CLP引入的位置错位恒定,当优化收益减弱时,代价占比上升,最终导致性能饱和。该收益-成本机制解释了PLT在两轮循环处的最优表现,并为循环数选择提供可解释依据。

原文摘要 · Abstract (English)

Looped Transformers scale latent computation by repeatedly applying shared blocks, but sequential looping increases latency and KV-cache memory with the loop count. Parallel loop Transformers (PLT) alleviate this cost through cross-loop position offsets (CLP) and shared-KV gated sliding-window attention, making loop count a practical design choice. We therefore study PLT loop-count selection through a gain--cost view: an extra loop may refine representations, but CLP also introduces a positional mismatch at each loop boundary. We instantiate this study by training LoopCoder-v2, a family of 7B PLT coders with different loop counts, from scratch on 18T tokens, followed by matched instruction tuning and evaluation. Empirically, the two-loop variant delivers broad gains over the non-looped baseline across code generation, code reasoning, agentic software engineering, and tool-use benchmarks, improving SWE-bench Verified from 43.0 to 64.4 points and Multi-SWE from 14.0 to 31.0 points. In contrast, variants with three or more loops regress, revealing a strongly non-monotonic loop-count effect. Our diagnostics show that loop 2 provides the main productive refinement, while later loops yield diminishing, oscillatory updates and reduced representational diversity. Because the CLP-induced mismatch remains roughly fixed as refinement gains shrink, the offset cost increasingly dominates. This gain--cost trade-off explains PLT's saturation at two loops and provides diagnostics for loop-count selection.

代码生成高效推理模型优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。