让大模型学会聚焦图像细节，通过自身经验提升全局理解能力。
Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation
- 用同一模型的局部视图与全图视图做自蒸馏，传递细节感知优势。
- 在多个细粒度视觉任务上表现优于更大模型或复杂推理系统。
- 无需外部标签或工具，训练即能提升模型对关键区域的注意力。
多模态大语言模型在细粒度视觉理解上仍存在困难，答案常依赖于图像中微小但关键的证据。我们发现存在区域到全局的感知差距：当模型基于包含证据的局部裁剪图回答问题时，准确率显著高于使用完整图像的情况，表明失败主因是难以聚焦相关区域，而非局部识别能力不足。为此，我们提出 Vision-OPD（视觉在线策略自蒸馏），一种从区域到全局的自蒸馏框架，将模型自身在局部区域上的优势转移至全图推理策略。Vision-OPD 从同一 MLLM 中构建两个条件策略：以裁剪图为输入的教师模型和以全图为输入的学生模型。学生生成在线推理轨迹，Vision-OPD 在这些轨迹上最小化教师与学生在下一个词分布上的分词级差异。该方法使模型在无需外部教师、真实标签、奖励验证器或推理时工具调用的情况下，内化视觉缩放带来的好处。在多个细粒度视觉理解基准测试中，Vision-OPD 模型的表现达到甚至超过更大规模的开源、闭源及‘图像思维’类智能体模型。代码已公开于 https://github.com/VisionOPD/Vision-OPD。
原文摘要 · Abstract (English)
Multimodal Large Language Models (MLLMs) still struggle with fine-grained visual understanding, where answers often depend on small but decisive evidence in the full image. We observe a regional-to-global perception gap: the same MLLM answers fine-grained questions more accurately when conditioned on evidence-centered crops than on the corresponding full images, suggesting that many failures stem from difficulty to focus on relevant evidence rather than insufficient local recognition ability. Motivated by this observation, we propose Vision-OPD (Vision On-Policy Distillation), a regional-to-global self-distillation framework that transfers the model's own privileged regional perception to its full-image policy. Vision-OPD instantiates two conditional policies from the same MLLM: a crop-conditioned teacher and a full-image-conditioned student. The student generates on-policy rollouts, and Vision-OPD minimizes token-level divergence between the teacher and student next-token distributions along these rollouts. This enables the model to internalize the benefit of visual zooming without external teacher models, ground-truth labels, reward verifiers, or inference-time tool use. Experiments on multiple fine-grained visual understanding benchmarks show that Vision-OPD models achieve competitive or superior performance against much larger open-source, closed-source, and "Thinking-with-Images" agentic models. The code is available at https://github.com/VisionOPD/Vision-OPD
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。