用冻结的预训练嵌入实现高效视觉语言理解，性能接近顶尖模型。
FrEVL: Leveraging Frozen Pretrained Embeddings for Efficient Vision-Language Understanding
- 冻结预训练视觉嵌入，仅需6840万参数即可达成95%性能
- 在标准基准上达到95%顶尖模型表现，计算速度提升2.3倍
- 适合输入可预先计算或对性能要求不苛刻的部署场景
视觉语言模型的部署受限于高昂的计算开销。本文提出FrEVL框架，探索冻结预训练嵌入是否能支持有效的视觉语言理解。分析表明，冻结嵌入包含丰富的判别性信息，在标准基准上仅用6840万可训练参数即达到95%的顶尖性能。这一性能差异揭示关键洞见：冻结嵌入的有效性取决于预训练目标与下游任务需求的一致性。在包含嵌入提取在内的端到端计算中，FrEVL实现2.3倍加速，能耗降低52%，适用于输入可预计算或部署约束优先于微小性能提升的场景。本研究为实践者提供选择冻结嵌入替代全模型部署的决策依据。我们将开源完整实现与评估框架，推动高效多模态理解研究。
原文摘要 · Abstract (English)
The deployment of vision-language models remains constrained by substantial computational requirements. We present \textbf{FrEVL}, a framework exploring whether frozen pretrained embeddings can support effective vision-language understanding. Our analysis reveals that frozen embeddings contain rich information for discriminative tasks, achieving 85\% to 95\% of state-of-the-art performance on standard benchmarks with only 68.4M trainable parameters. This performance dichotomy reveals a critical insight: frozen embedding effectiveness depends on alignment between pretraining objectives and downstream task requirements. When accounting for end-to-end computation including embedding extraction, FrEVL provides $2.3\times$ speedup with 52\% lower energy consumption, making it suitable for scenarios with pre-computable inputs or when deployment constraints outweigh marginal performance gains. Our evaluation provides practitioners with guidance on when frozen embedding approaches represent viable alternatives to full model deployment. We will release our complete implementation and evaluation framework to facilitate further research into efficient multi-modal understanding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。