提升视觉语言模型对图像细节的依赖，让学生更听从视觉信号。
Visual-Advantage On-Policy Distillation for Vision-Language Models
- 基于视觉优势设计新蒸馏目标，聚焦关键视觉令牌
- 在多个数学数据集上显著优于传统方法，效果随教师规模和数据量增长
- 适合需要强化视觉理解能力的多模态模型训练
在语言模型中，就地知识蒸馏已被证明有效，但在视觉语言模型（VLMs）中的应用仍不充分。我们发现，标准就地蒸馏虽能提升学生输出质量，却未能增强其对视觉输入的依赖：在视觉关键标记上，无论是否包含细粒度视觉信息，学生的预测基本不变，而教师的预测则高度依赖该信息。为此，我们引入视觉优势（VA），即教师在有无细粒度视觉信息条件下对学生生成轨迹的逐标记对数概率差异。VA集中在少数标记上，这些高VA标记才是真正承载视觉监督信号的。我们提出视觉优势就地蒸馏（VA-OPD），在两个粒度上使用VA：轨迹平均VA进行整体重加权，以及在高VA与低VA组内分别计算令牌级KL散度。在两个数学数据集（Geometry3K 和 ViRL39K）上训练，并在八个涵盖数学推理与视觉理解的基准上评估，覆盖三个教师规模（4B、8B、32B）的Qwen3-VL系列模型。结果表明，VA-OPD在所有基准上均优于标准就地蒸馏，且性能增益随教师规模与数据规模增加而单调上升，表明两者具有持续正向叠加效应。
原文摘要 · Abstract (English)
On-policy knowledge distillation has proven effective for language models, yet its application to vision-language models (VLMs) remains underexplored. We observe that standard on-policy distillation can improve a student's output quality while failing to strengthen its reliance on visual input: on vision-critical tokens, the student's predictions remain largely unchanged whether or not fine-grained visual detail is present, even though the teacher's predictions depend heavily on it.To make this difference observable, we introduce visual advantage (VA), the token-level log-probability difference when the teacher scores a student-generated rollout with versus without access to fine-grained visual detail. VA is concentrated in a small minority of tokens, and these high-VA tokens are the ones that actually carry the visual supervision signal. This motivates a distillation objective that treats them differently from language scaffolding, so their contribution is not diluted by the abundant surrounding language tokens.We propose Visual-Advantage On-Policy Distillation (VA-OPD), which uses VA at two granularities: rollout-level reweighting by trajectory-averaged VA, and token-level KL averaged within high-VA and low-VA groups separately. We train on two math datasets (Geometry3K and ViRL39K) and evaluate on eight benchmarks covering both mathematical reasoning and visual understanding, across three teacher sizes (4B, 8B, and 32B) on the Qwen3-VL family. VA-OPD improves over standard on-policy distillation on every benchmark, with the gain growing monotonically along both the teacher-size and data-scale axes, suggesting that these factors compound consistently.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。