用强化学习动态调整文本生成中的分布对齐方式，提升模型质量和泛化能力。
ARKD: Adaptive Reinforcement Learning-Guided Bidirectional KL Divergence Distillation for Text Generation
- 通过强化学习动态分配前向与反向KL散度权重，实现主模式与长尾分布的双重对齐
- 在Rouge-L和BertScore上相比贪心方法提升0.4-0.6分，多基准测试表现更优
- 适用于需要高质量、高泛化性的轻量化大模型文本生成场景
知识蒸馏（KD）是压缩大语言模型的关键技术，但依赖单一KL目标的方法难以平衡主分布拟合与长尾概率建模，限制了生成质量与泛化能力。本文从理论与实证角度分析前向与反向KL散度（FKL/RKL）在分布对齐中的互补作用。提出一种基于强化学习的自适应KL加权蒸馏框架，其中策略网络根据教师-学生分布特征动态分配FKL与RKL权重，通过即时奖励信号引导双模式对齐。大量实验表明，该方法在Rouge-L与BertScore指标上持续提升，优于贪心启发式方法0.4–0.6分，并在多个基准测试中超越其他基线方法。
原文摘要 · Abstract (English)
Knowledge distillation (KD) is a key technique for compressing Large Language Models (LLMs), yet methods relying on a single KL objective often fail to balance primary distribution fitting with long-tail probability modeling, limiting both generation quality and generalization. To address this, we analyze the complementary roles of forward and reverse KL divergence (FKL/RKL) in distribution alignment from theoretical and empirical perspectives. We then propose a reinforcement-learning-based adaptive KL-weighted distillation framework, in which a policy network dynamically assigns weights to FKL and RKL based on teacher-student distributional characteristics, guided by immediate reward signals to achieve dual alignment on principal and long-tail modes. Extensive experiments demonstrate consistent improvements across Rouge-L and BertScore metrics, surpassing greedy heuristics by 0.4-0.6 points and outperforming other baseline methods on diverse benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。