用可学习的软提示实现多任务在线蒸馏，让大模型高效学新技能不遗忘。
One Student, Many Teachers: Multi-Task On-Policy Distillation via Soft-Prompt Privileged Context
- 教师仅通过可学习软提示与学生区分，冻结主干网络保持表征一致。
- 单任务下性能接近全量微调，参数量训练减少数个数量级。
- 多任务并行蒸馏，平均得分56.2，且不损害通用能力，适合快速适配新任务。
在线策略自蒸馏（OPSD）通过共享主干的教师模型监督自身生成过程，帮助大语言模型学习新技能。现有方法或在输入端注入特权信息（导致事后解释），或微调权重，造成任务间漂移与遗忘。本文提出 extmethod，其教师仅通过一个可学习的软提示与学生不同：在冻结主干的情况下，基于$(x, y_ ext{gold})$对训练提示，生成任务特定教师，保留学生完全相同的表征几何结构。该方法天然适用于多任务场景，将混合语料中每条样本路由至对应软提示教师，使单一学生同时吸收$K$个教师的知识；推理时所有提示被丢弃。在Qwen3-1.7B-Base和Phi-4-mini-instruct上，四个任务（科学、工具使用、生物、数学）测试中，单任务版本（带预训练教师的OPD）性能匹配或超越全量微调，训练参数量减少数个数量级；多任务版本取得最高平均分（56.2），同时维持通用能力基准——而顺序SFT则显著下降。
原文摘要 · Abstract (English)
On-policy self-distillation (OPSD) teaches large language models new skills through a teacher that shares the student's backbone and supervises its own rollouts. Existing teachers either inject privileged context at the input -- inducing post-hoc rationalization -- or fine-tune weights, accumulating drift and forgetting across tasks. We propose \method, whose teacher differs from the student only by a learnable soft prompt: trained on $(x, y_\text{gold})$ pairs with the backbone frozen, the prompt yields a task-specific teacher that preserves the student's exact representational geometry. \method\ extends naturally to multi-task settings by routing each example in a merged corpus to its corresponding soft-prompt teacher, allowing a single student to absorb knowledge from $K$ teachers in parallel; at inference, all prompts are discarded. On Qwen3-1.7B-Base and Phi-4-mini-instruct across four tasks (Science, Tool Use, Biology, Math), the single-task variant (OPD with a PT teacher) matches or exceeds full fine-tuning while training orders of magnitude fewer parameters, and the multi-task variant achieves the best overall average ($56.2$ on Qwen3-1.7B-Base) while preserving general-capability benchmarks -- in contrast to sequential SFT, which degrades both.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。