arXiv:2512.22238cs.LGcs.AI2025-12被引 6

通过渐进式掩码与强化学习,让小模型更稳定地学大模型的多模态能力。

Masking Teacher and Reinforcing Student for Distilling Vision-Language Models

  • 用渐进解封教师模型权重,降低知识迁移复杂度。
  • 结合准确率与可迁移性奖励,提升学生模型性能。
  • 适合资源受限场景下高效部署高性能多模态模型。

大规模视觉语言模型(VLMs)虽在多模态理解上表现卓越,但其庞大体积难以在移动端或边缘设备部署。为实现紧凑而强大的VLMs,需高效从大教师模型中学习。然而,因师生模型规模差距大,学生常无法复现教师复杂的高维表征,导致学习不稳定、性能下降。为此,我们提出Masters(Masking Teacher and Reinforcing Student),一种基于掩码渐进与强化学习的知识蒸馏框架。该方法先对教师模型非主导权重进行掩码以减少冗余复杂度,随后在训练中逐步恢复教师容量。这一策略使学生能平稳、稳定地学习到更丰富的表征。为进一步优化知识转移,框架引入离线强化学习阶段,采用两种互补奖励:准确率奖励衡量生成答案正确性,蒸馏奖励量化教师到学生响应的可迁移性。相比计算开销大、输出冗长的在线思考-回答强化学习范式,本方法利用掩码教师预生成的回答,提供丰富而高效的指导,使学生无需经历思考过程即可达到优异性能。

原文摘要 · Abstract (English)

Large-scale vision-language models (VLMs) have recently achieved remarkable multimodal understanding, but their massive size makes them impractical for deployment on mobile or edge devices. This raises the need for compact yet capable VLMs that can efficiently learn from powerful large teachers. However, distilling knowledge from a large teacher to a small student remains challenging due to their large size gap: the student often fails to reproduce the teacher's complex, high-dimensional representations, leading to unstable learning and degraded performance. To address this, we propose Masters (Masking Teacher and Reinforcing Student), a mask-progressive reinforcement learning (RL) distillation framework. Masters first masks non-dominant weights of the teacher to reduce unnecessary complexity, then progressively restores the teacher by gradually increasing its capacity during training. This strategy allows the student to learn richer representations from the teacher in a smooth and stable manner. To further refine knowledge transfer, Masters integrates an offline RL stage with two complementary rewards: an accuracy reward that measures the correctness of the generated responses, and a distillation reward that quantifies the ease of transferring responses from teacher to student. Unlike online think-answer RL paradigms that are computationally expensive and generate lengthy responses, our offline RL leverages pre-generated responses from masked teachers. These provide rich yet efficient guidance, enabling students to achieve strong performance without requiring the think-answer process.

知识蒸馏多模态强化学习模型压缩

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。