arXiv:2512.20839cs.CV2025-12

根据图像内容动态调整预处理,让视觉语言模型更快更省资源。

Input-Adaptive Visual Preprocessing for Efficient Fast Vision-Language Model Inference

  • 按图像复杂度自动调节分辨率和裁剪区域,减少冗余计算。
  • 推理时间减少超50%,视觉令牌数降低55%以上。
  • 无需改架构或重训练,适合追求部署效率的研究者。

视觉语言模型在多模态推理任务中表现优异,但高分辨率输入导致推理延迟高、计算成本大。尽管FastVLM等架构通过优化视觉编码器提升效率,现有流程仍依赖静态预处理,对简单图像造成冗余计算。本文提出一种自适应视觉预处理方法,基于图像内容特征动态调整输入分辨率与空间覆盖范围。该方法结合内容感知分析、自适应分辨率选择和内容感知裁剪,在视觉编码前消除视觉冗余。重要的是,该方法可无缝集成至FastVLM,无需修改其架构或重新训练。我们在DocVQA数据集的一个子集上进行纯推理评估,聚焦效率指标。结果表明,自适应预处理使单图推理时间减少超过50%,平均生成时间下降,视觉令牌数持续降低55%以上。这些发现证明,输入感知预处理是提升视觉语言模型部署效率的有效且轻量策略。为保障可复现性,我们已将实现作为FastVLM仓库的分叉版本发布,代码地址:https://github.com/kmdavidds/mlfastlm。

原文摘要 · Abstract (English)

Vision-Language Models (VLMs) have demonstrated strong performance on multimodal reasoning tasks, but their deployment remains challenging due to high inference latency and computational cost, particularly when processing high-resolution visual inputs. While recent architectures such as FastVLM improve efficiency through optimized vision encoders, existing pipelines still rely on static visual preprocessing, leading to redundant computation for visually simple inputs. In this work, we propose an adaptive visual preprocessing method that dynamically adjusts input resolution and spatial coverage based on image content characteristics. The proposed approach combines content-aware image analysis, adaptive resolution selection, and content-aware cropping to reduce visual redundancy prior to vision encoding. Importantly, the method is integrated with FastVLM without modifying its architecture or requiring retraining. We evaluate the proposed method on a subset of the DocVQA dataset in an inference-only setting, focusing on efficiency-oriented metrics. Experimental results show that adaptive preprocessing reduces per-image inference time by over 50\%, lowers mean full generation time, and achieves a consistent reduction of more than 55\% in visual token count compared to the baseline pipeline. These findings demonstrate that input-aware preprocessing is an effective and lightweight strategy for improving deployment-oriented efficiency of vision-language models. To facilitate reproducibility, our implementation is provided as a fork of the FastVLM repository, incorporating the files for the proposed method, and is available at https://github.com/kmdavidds/mlfastlm.

视觉语言模型高效推理自适应预处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。