arXiv:2510.10022cs.CV2025-10被引 4

用轻量查询适配器高效提取视频相关文本特征,兼顾效果与参数效率。

Q-Adapter: Visual Query Adapter for Extracting Textually-related Features in Video Captioning

  • 在视觉编码器中引入可学习查询令牌和门控层,实现稀疏特征提取。
  • 在MSR-VTT和MSVD上超越所有参数高效方法,达最佳性能。
  • 仅需1.4%参数量,适合资源受限场景下的视频描述任务。

视频描述的最新进展依赖于大规模预训练模型,通常采用“预训练后微调”范式,但随着模型规模增大,计算开销显著增加。参数高效微调(PEFT)提供了一种替代方案,但主要聚焦于多模态大语言模型(MLLMs)的语言部分,对微调过程中视觉信息的理解仍不充分。为此,我们提出查询适配器(Q-Adapter),一种轻量级视觉适配模块,通过在视觉编码器中引入可学习查询令牌和门控层,实现无需外部文本监督的稀疏、描述相关特征提取。我们在MSR-VTT和MSVD两个主流数据集上评估Q-Adapter,结果表明其在BLEU@4、METEOR、ROUGE-L和CIDEr指标上均优于现有参数高效方法,并在仅使用1.4%参数量的情况下,达到与全量微调方法相当的性能。我们还分析了关键超参数与设计选择的影响,为适配器学习优化提供指导。这些结果证明Q-Adapter在生成质量与参数效率之间实现了良好平衡,具备在视频-语言建模中规模化应用的潜力。

原文摘要 · Abstract (English)

Recent advances in video captioning are driven by large-scale pretrained models, which follow the standard "pre-training followed by fine-tuning" paradigm, where the full model is fine-tuned for downstream tasks. Although effective, this approach becomes computationally prohibitive as the model size increases. The Parameter-Efficient Fine-Tuning (PEFT) approach offers a promising alternative, but primarily focuses on the language components of Multimodal Large Language Models (MLLMs). Despite recent progress, PEFT remains underexplored in multimodal tasks and lacks sufficient understanding of visual information during fine-tuning the model. To bridge this gap, we propose Query-Adapter (Q-Adapter), a lightweight visual adapter module designed to enhance MLLMs by enabling efficient fine-tuning for the video captioning task. Q-Adapter introduces learnable query tokens and a gating layer into Vision Encoder, enabling effective extraction of sparse, caption-relevant features without relying on external textual supervision. We evaluate Q-Adapter on two well-known video captioning datasets, MSR-VTT and MSVD, where it achieves state-of-the-art performance among the methods that take the PEFT approach across BLEU@4, METEOR, ROUGE-L, and CIDEr metrics. Q-Adapter also achieves competitive performance compared to methods that take the full fine-tuning approach while requiring only 1.4% of the parameters. We further analyze the impact of key hyperparameters and design choices on fine-tuning effectiveness, providing insights into optimization strategies for adapter-based learning. These results highlight the strong potential of Q-Adapter in balancing caption quality and parameter efficiency, demonstrating its scalability for video-language modeling.

视频描述适配器参数效率多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。