利用图像分辨率差异实现多模态大模型的高效自蒸馏。
RP-OPSD: Resolution-Privileged On-Policy Self-Distillation for Multimodal Large Language Models

- 用高低分辨率图像差作为教师监督信号,无需额外标注或外部模型。
- 在原分辨率下性能提升5.45%,训练速度提高1.78倍。
- 适合追求高效训练与推理优化的多模态大模型研究者。
在策略自蒸馏(OPSD)中,教师利用学生无法访问的特权信息提供密集的逐标记监督。现有方法通常依赖已验证的解题路径、外部模型生成的解释或人工标注的视觉证据,限制了其在多模态大语言模型中的可扩展性。为此,我们利用同一图像高、低分辨率之间的信息差距,提出分辨率特权在策略自蒸馏(RP-OPSD)。训练时,学生从四分之一原始分辨率的图像生成策略轨迹,教师则使用全分辨率图像提供监督。通过最小化两者在学生轨迹上的输出分布差异,学生学习到在高分辨率输入下教师的预测行为,从而增强低分辨率下的表现,并将改进效果迁移至原始分辨率推理。RP-OPSD无需额外人工标注或外部模型,仅需图像-问题对即可实现。在Qwen3.5-9B上的实验表明,该方法在原始分辨率下平均性能相对提升5.45%,训练速度比OPSD快1.78倍。结果证明,分辨率差异可作为简单且可扩展的特权信息源,为多模态大语言模型提供高效可靠的在策略自蒸馏方案。
原文摘要 · Abstract (English)
On-Policy Self-Distillation (OPSD) uses privileged information available only to the teacher to provide dense token-level supervision on trajectories generated by the student. However, existing methods often rely on verified solution traces, explanations generated by external models, or manually localized visual evidence, which limits their scalable application to multimodal large language models. To address this issue, we exploit the information gap between high- and low-resolution views of the same image and propose RP-OPSD (Resolution-Privileged On-Policy Self-Distillation for Multimodal Large Language Models). During training, the student policy generates on-policy trajectories from images at one-quarter of the original resolution, while the teacher policy provides supervision using the original-resolution images. By minimizing the divergence between their output distributions along the student trajectories, the student learns the predictive behavior of the teacher under high-resolution inputs, thereby strengthening its low-resolution capability and transferring the learned improvement to original-resolution inference. RP-OPSD requires neither additional human annotations nor external models to generate solution traces but only image--question pairs. Experiments on Qwen3.5-9B show that RP-OPSD achieves a 5.45\% relative improvement in average performance at the original resolution and a $1.78\times$ training speedup over OPSD. These results demonstrate that resolution differences can serve as a simple and scalable source of privileged information, providing an effective and efficient approach to on-policy self-distillation for multimodal large language models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。