arXiv:2602.11858cs.CVcs.AI2026-02被引 28

让大模型不用反复缩放也能看清细节,训练时就学会‘聚焦观察’。

Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception

  • 用教师模型在局部区域生成高质量问答数据,蒸馏到学生模型中
  • 在845个样本的ZoomBench上,细粒度感知性能超越现有方法
  • 适合需要快速精准识别细节的场景,如医学影像、精密检测

多模态大语言模型(MLLM)在整体视觉理解上表现优异,但在细粒度感知任务中仍面临挑战——关键证据微小且易被全局上下文掩盖。现有“以图思考”方法通过推理时反复缩放感兴趣区域来缓解此问题,但因多次调用工具和重新编码导致高延迟。为此,我们提出区域到图像蒸馏(Region-to-Image Distillation),将缩放这一推理过程转化为训练阶段的构建模块,使模型在单次前向传播中即可内化“智能缩放”的优势。具体而言,先对微裁剪区域进行缩放,由强教师模型生成高质量视觉问答数据,再将这些区域相关的监督信号蒸馏回全图模型。训练后,学生模型无需工具调用即可实现“一眼看清”的细粒度感知能力。为严谨评估该能力,我们进一步构建了包含845个样本的混合标注基准ZoomBench,涵盖六个细粒度感知维度,并提出双视图协议量化全局与区域之间的“缩放差距”。实验表明,所提模型在多个细粒度感知基准上达到领先性能,同时在视觉推理和GUI代理等任务中也提升了通用多模态认知能力。我们还探讨了何时需要“以图思考”,何时其优势可被蒸馏至单次前向传播。代码已开源:https://github.com/inclusionAI/Zooming-without-Zooming。

原文摘要 · Abstract (English)

Multimodal Large Language Models (MLLMs) excel at broad visual understanding but still struggle with fine-grained perception, where decisive evidence is small and easily overwhelmed by global context. Recent "Thinking-with-Images" methods alleviate this by iteratively zooming in and out regions of interest during inference, but incur high latency due to repeated tool calls and visual re-encoding. To address this, we propose Region-to-Image Distillation, which transforms zooming from an inference-time tool into a training-time primitive, thereby internalizing the benefits of agentic zooming into a single forward pass of an MLLM. In particular, we first zoom in to micro-cropped regions to let strong teacher models generate high-quality VQA data, and then distill this region-grounded supervision back to the full image. After training on such data, the smaller student model improves "single-glance" fine-grained perception without tool use. To rigorously evaluate this capability, we further present ZoomBench, a hybrid-annotated benchmark of 845 VQA data spanning six fine-grained perceptual dimensions, together with a dual-view protocol that quantifies the global--regional "zooming gap". Experiments show that our models achieve leading performance across multiple fine-grained perception benchmarks, and also improve general multimodal cognition on benchmarks such as visual reasoning and GUI agents. We further discuss when "Thinking-with-Images" is necessary versus when its gains can be distilled into a single forward pass. Our code is available at https://github.com/inclusionAI/Zooming-without-Zooming.

细粒度感知模型蒸馏多模态视觉问答

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。