突破分词器限制，实现跨模型家族的高效知识蒸馏
Breaking the Tokenizer Barrier: On-Policy Distillation across Model Families
- 提出精确分词映射算法，让不同分词器间也能进行原策略蒸馏
- 跨分词器蒸馏比基线方法节省大量计算资源，性能更优
- 适合想复用不同模型家族知识的研究者和开发者
原策略蒸馏（OPD）已成为大语言模型后训练的核心技术，用于将领域专家模型的知识迁移至学生模型。然而现有方法要求教师与学生模型使用相同分词器，限制了OPD在模型系列外的应用。当前主流做法通常对教师生成的响应进行监督微调（SFT）以实现跨分词器蒸馏，但无法捕捉教师概率分布中的丰富知识。本文通过精确的分词映射算法，使标准原策略蒸馏可跨模型家族运行，确保高保真词级信号在不同分词器间传递。大量实验表明，跨分词器OPD在多个基准测试上显著优于基线，计算效率更高。本工作拓展了OPD的师生模型组合范围，为大模型间的知识交互开辟新路径。
原文摘要 · Abstract (English)
On-Policy Distillation (OPD) has become a core technique in the post-training of Large Language Models (LLMs) for transferring knowledge from domain experts to student models. However, existing OPD distillation methods require teacher and student models to share the same tokenizer, restricting the applicability of OPD within the model series. Current mainstream practice typically employs Supervised Fine-Tuning (SFT) on teacher-generated responses for cross-tokenizer distillation, which fails to capture the rich knowledge embedded in the teacher's probability distribution. In this work, we enable the standard on-policy distillation method to operate across model families, ensuring that high-fidelity token-level signals can propagate across different tokenizers with a precise token-mapping algorithm. Extensive experiments show that cross-tokenizer OPD is significantly more compute-efficient than baselines on various benchmarks. Our results unlock a broader range of teacher-student pairs for OPD, opening up new avenues for adapting and enhancing interactions between LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。