按代码结构粒度选择关键块进行监督,大幅减少训练标记量
CODEBLOCK: Learning to Supervise Code at the Right Granularity

- 以语法完整代码块为单位选参,避免破坏程序结构
- 仅用1.9%的标记实现比全令牌训练更好的平均通过率
- 适合追求高效微调的代码大模型研究者与开发者
代码大模型的有监督微调通常对所有输出标记统一应用交叉熵损失,隐含假设每个标记提供等量学习信号。近期的令牌级选择方法在自然语言微调中通过只监督高价值令牌挑战这一假设。然而,直接将令牌级掩码应用于代码会破坏语法和语义完整的程序单元,因为代码依赖结构完整性与定义-使用关系。为此,我们提出 CodeBlock:一种结构感知的稀疏监督框架,选择结构完整的代码证据而非孤立标记。CodeBlock 首先筛选高质量指令-响应对,将代码响应分割为语法一致的代码单元,通过核心逻辑标记上广义交叉熵聚合估计其价值,并结合数据流可达性与桥接信号重排序,优先选择能传播或连接重要程序依赖的代码块。训练时,完整响应仍作为上下文,损失仅作用于选定代码块及有信息量的自然语言标记。在六个代码生成基准上的实验表明,CodeBlock 在平均 pass@1 上优于全令牌微调和竞争性选择基线,且仅使用1.9%的受监督响应标记。
原文摘要 · Abstract (English)
Supervised fine-tuning of code LLMs typically applies uniform cross-entropy loss to all response tokens, implicitly assuming that every token provides equally useful learning signal. Recent token-level selection methods challenge this assumption in natural-language SFT by supervising only high-value tokens. However, directly transferring token-level masking to code can break syntactically and semantically coherent program units, because code depends on structural completeness and definition-use relations. We therefore propose CodeBlock, a structure-aware sparse supervision framework that selects structure-complete code evidence rather than isolated tokens. CodeBlock first selects high-quality instruction-response pairs, then partitions code responses into syntactically coherent coding items, estimates their utility by aggregating generalized cross-entropy over core logic tokens, and reranks them with data-flow reach and bridge signals to prioritize blocks that propagate or connect important program dependencies. During training, the full response remains available as context, while loss is applied only to selected code items and informative natural-language tokens. Experiments on six code-generation benchmarks show that CodeBlock achieves stronger average pass@1 than full-token SFT and competitive selection baselines, while using only 1.9% of supervised response tokens.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。