解决跨分词器教学中信号丢失问题,提升小模型学习效果。
SimCT: Recovering Lost Supervision for Cross-Tokenizer On-Policy Distillation

- 用多词连续片段替代单字匹配,扩大监督信号范围
- 在数学推理与代码生成任务上显著优于传统方法
- 适合跨分词器场景下知识迁移,尤其小模型训练
在线策略蒸馏(OPD)是将教师模型行为迁移到小型学生模型的标准方法,但其隐含假设是教师与学生在词粒度上预测可直接对比,这一假设在分词器不一致时失效。当模型使用不同分词器时,精确共享词匹配会无声丢弃大量教师信号,尤其是在词汇不一致的位置。本文提出简单的跨分词器在线策略蒸馏(SimCT),通过在共享词之外,引入两个分词器都能识别的短多词连续片段作为监督单元,恢复被丢弃的信号,同时保持原始蒸馏损失形式不变。我们证明这些片段是可联合分词的最细粒度监督接口;更粗粒度的替代方案会消除对在线学习有用的师生差异。在三个数学推理与代码生成任务上的异构教师-学生对中,SimCT consistently优于共享词汇蒸馏和代表性跨分词器基线,消融实验确认性能提升源于恢复了因精确共享词匹配而被丢弃的监督信号。代码已开源。
原文摘要 · Abstract (English)
On-policy distillation (OPD) is a standard tool for transferring teacher behavior to a smaller student, but it implicitly assumes that teacher and student predictions are comparable token by token, an assumption that fails whenever the two models tokenize the same text differently. Under heterogeneous tokenizers, exact shared-token matching silently discards a large fraction of the teacher signal at precisely the positions where vocabularies disagree. We propose \textbf{\underline{Sim}ple \underline{C}ross-\underline{T}okenizer OPD (SimCT)}, which restores this signal by enlarging the supervision space: alongside shared tokens, SimCT compares teacher and student over short multi-token continuations that both tokenizers can realize, leaving the OPD loss form itself unchanged. We show that these units are the finest jointly tokenizable supervision interface, and that coarser alternatives remove teacher-student distinctions that are useful for on-policy learning. Across three heterogeneous teacher-student pairs on mathematical reasoning and code-generation benchmarks, SimCT shows consistent gains over shared-vocabulary OPD and representative cross-tokenizer baselines, with ablations confirming that the improvements come from recovering supervision discarded by exact shared-token matching. Code is available at \href{https://github.com/sunjie279/SimCT-}{https://github.com/sunjie279/SimCT-}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。