用对抗机制让小模型在不看大模型内部的情况下，学出接近的效果。
Black-Box On-Policy Distillation of Large Language Models
- 把学生模型当生成器，用判别器区分其输出与教师模型差异，形成对抗训练。
- 在LMSYS-Chat评测中，140亿参数的学生模型效果接近GPT-5-Chat。
- 适合无法访问教师模型内部参数的场景，如商业闭源模型蒸馏。
黑盒蒸馏仅通过教师大语言模型的文本输出来训练学生模型，无需访问其内部激活值或参数。本文提出生成对抗蒸馏（GAD），实现无监督、黑盒的在线策略蒸馏。GAD将学生模型视为生成器，训练一个判别器以区分其输出与教师模型响应，构成极小极大博弈。判别器作为在线策略奖励模型，与学生共同演化，提供稳定且自适应的反馈。实验表明，GAD持续优于常见的序列级知识蒸馏方法。具体而言，使用GAD训练的Qwen2.5-14B-Instruct学生模型，在LMSYS-Chat自动评估中达到与教师模型GPT-5-Chat相当的性能。结果验证了GAD作为黑盒大模型蒸馏的有效且有前景的新范式。
原文摘要 · Abstract (English)
Black-box distillation creates student large language models (LLMs) by learning from a proprietary teacher model's text outputs alone, without access to its internal logits or parameters. In this work, we introduce Generative Adversarial Distillation (GAD), which enables on-policy and black-box distillation. GAD frames the student LLM as a generator and trains a discriminator to distinguish its responses from the teacher LLM's, creating a minimax game. The discriminator acts as an on-policy reward model that co-evolves with the student, providing stable, adaptive feedback. Experimental results show that GAD consistently surpasses the commonly used sequence-level knowledge distillation. In particular, Qwen2.5-14B-Instruct (student) trained with GAD becomes comparable to its teacher, GPT-5-Chat, on the LMSYS-Chat automatic evaluation. The results establish GAD as a promising and effective paradigm for black-box LLM distillation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。