arXiv:2510.02750cs.CV2025-10被引 2

用贝叶斯方法在测试时动态调整视觉语言模型的类别先验和特征,提升真实场景下的识别与检测性能。

Bayesian Test-time Adaptation for Object Recognition and Detection with Vision-language Models

  • 基于贝叶斯推断,通过动态缓存更新类别嵌入、空间尺度和先验分布。
  • 在多个数据集上实现比现有方法更高的准确率,识别任务最高提升3.2%。
  • 无需反向传播,适合实时部署,特别适合边缘设备上的模型适应。

视觉语言模型(如CLIP和Grounding DINO)在目标识别与检测任务中表现卓越,但在实际分布偏移下性能下降明显。测试时自适应(TTA)旨在推理过程中动态调整模型。现有方法或依赖计算量大的反向传播,难以实时部署;或仅关注似然适配,忽视先验的重要性。我们此前提出的贝叶斯类别适配(BCA)解决了识别任务中的这些问题,提出无需训练的框架并引入自适应先验。本文在此基础上提出统一的训练自由框架BCA+,适用于识别与检测任务。BCA+引入动态缓存,自适应存储并更新类别嵌入、空间尺度(检测任务)及由历史预测推导出的自适应类别先验。将适配建模为贝叶斯推断问题,最终预测融合初始VLM输出与基于缓存的预测。该缓存预测结合动态更新的似然(度量特征与尺度相似性)与先验(反映类别分布演化)。双适应机制配合不确定性引导融合,可同时修正模型语义理解与上下文置信度。作为无需反向传播的训练自由方法,BCA+高效且适用于实时场景。大量实验表明,BCA+在识别与检测基准上均达到当前最优性能。

原文摘要 · Abstract (English)

Vision-language models (VLMs) such as CLIP and Grounding DINO have achieved remarkable success in object recognition and detection. However, their performance often degrades under real-world distribution shifts. Test-time adaptation (TTA) aims to mitigate this issue by adapting models during inference. Existing methods either rely on computationally expensive backpropagation, which hinders real-time deployment, or focus solely on likelihood adaptation, which overlooks the critical role of the prior. Our prior work, Bayesian Class Adaptation (BCA), addressed these shortcomings for object recognition by introducing a training-free framework that incorporates adaptive priors. Building upon this foundation, we now present Bayesian Class Adaptation plus (BCA+), a unified, training-free framework for TTA for both object recognition and detection. BCA+ introduces a dynamic cache that adaptively stores and updates class embeddings, spatial scales (for detection), and, crucially, adaptive class priors derived from historical predictions. We formulate adaptation as a Bayesian inference problem, where final predictions are generated by fusing the initial VLM output with a cache-based prediction. This cache-based prediction combines a dynamically updated likelihood (measuring feature and scale similarity) and a prior (reflecting the evolving class distribution). This dual-adaptation mechanism, coupled with uncertainty-guided fusion, enables BCA+ to correct both the model's semantic understanding and its contextual confidence. As a training-free method requiring no backpropagation, BCA+ is highly efficient. Extensive experiments demonstrate that BCA+ achieves state-of-the-art performance on both recognition and detection benchmarks.

视觉语言模型测试时自适应贝叶斯方法实时推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。