轻量级多模态模型BcQLM仅12亿参数，实现高效视觉问答。
BcQLM: Efficient Vision-Language Understanding with Distilled Q-Gated Cross-Modal Fusion
- 用压缩版BreezeCLIP结合门控融合机制，降低计算开销。
- 1.2亿参数下性能接近主流大模型，跨数据集表现稳定。
- 适合边缘设备部署，模块化设计易拓展至其他多模态任务。
随着多模态大语言模型（MLLMs）的发展，其庞大的架构给资源受限环境的部署带来挑战。在注重能效、计算可扩展性与环境可持续性的大模型时代，开发轻量且高性能的模型对实际应用至关重要。为此，我们提出一种面向端到端视觉问答的轻量级MLLM框架。该方法基于一个紧凑但强大的视觉语言编码器BreezeCLIP，专为高效多模态理解优化。整体模型仅含12亿参数，显著降低计算成本，同时达到与标准规模MLLM相当的性能。在多个数据集上的实验进一步验证了其在精度与效率之间的良好平衡。模块化和可扩展设计使其能推广至更广泛的多模态任务。所提出的轻量级视觉语言框架命名为BcQLM（BreezeCLIP增强的Q-Gated多模态语言模型），为实际硬件约束下的可部署MLLM提供了可行路径。源代码见https://github.com/thico0224/BcQLM。
原文摘要 · Abstract (English)
As multimodal large language models (MLLMs) advance, their large-scale architectures pose challenges for deployment in resource-constrained environments. In the age of large models, where energy efficiency, computational scalability and environmental sustainability are paramount, the development of lightweight and high-performance models is critical for real-world applications. As such, we propose a lightweight MLLM framework for end-to-end visual question answering. Our proposed approach centres on BreezeCLIP, a compact yet powerful vision-language encoder optimised for efficient multimodal understanding. With only 1.2 billion parameters overall, our model significantly reduces computational cost while achieving performance comparable to standard-size MLLMs. Experiments conducted on multiple datasets further validate its effectiveness in balancing accuracy and efficiency. The modular and extensible design enables generalisation to broader multimodal tasks. The proposed lightweight vision-language framework is denoted as BcQLM (BreezeCLIP-enhanced Q-Gated Multimodal Language Model). It offers a promising path toward deployable MLLMs under practical hardware constraints. The source code is available at https://github.com/thico0224/BcQLM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。