arXiv:2606.12925cs.CVcs.LG2026-06中稿 · ICML被引 1

让冻结的视觉语言模型在分布偏移下更准地识别多标签,通过贝叶斯推理修正标签依赖关系。

Multi-Label Test-Time Adaptation with Bayesian Conditional Priors

论文配图:Multi-Label Test-Time Adaptation with Bayesian Conditional Priors
图 1 · 摘自论文原文
  • 用贝叶斯方法在线估计标签条件先验,无需微调主干网络
  • 在CLIP上使平均mAP提升至69.22(原57.31)和71.79(原62.61)
  • 适合无标注测试数据的多标签分类任务,尤其适用于分布偏移场景

使用冻结的视觉语言模型进行多标签识别在分布偏移下表现脆弱:标准零样本推理独立评分,忽略标签共现结构,导致主导概念压制兼容弱标签。本文提出贝叶斯条件先验(BCP)估计,一种无需梯度的测试时适应方法,在不调整主干网络的前提下引入标签依赖。将零样本逻辑视为固定图像-文本似然下的边缘后验代理,并将偏移引起的误差归因于标签先验不匹配。对每张测试图像,选择高置信度锚定标签,进行锚定条件贝叶斯修正。该更新在逻辑空间中为闭式解,具备点互信息(PMI)解释,显式促进兼容标签、抑制不兼容标签。通过轻量级二阶共现统计在线估计锚定条件先验,无需目标标注,额外开销仅一次前向传播。在多个标准多标签基准与多种CLIP主干上,BCP始终优于强基线,例如将RN50平均mAP从57.31提升至69.22,ViT-B/16从62.61提升至71.79。

原文摘要 · Abstract (English)

Multi-label recognition with frozen Vision-Language Models (VLMs) is brittle under distribution shift: standard zero-shot inference scores labels independently, ignoring co-occurrence structure and producing incoherent label sets where dominant concepts suppress weaker but compatible labels. We introduce Bayesian Conditional Priors (BCP) Estimation, a gradient-free test-time adaptation method that injects label dependency without tuning the backbone. BCP views zero-shot logits as a proxy for marginal posteriors under a fixed image-text likelihood and attributes shift-induced errors mainly to a mismatched label prior. For each test image, it selects a high-confidence anchor label and applies an anchor-conditioned Bayesian refinement. This update is closed-form in logit space and admits a pointwise mutual information (PMI) interpretation, explicitly promoting compatible labels and suppressing incompatible ones. BCP operates without target annotations by estimating anchor-conditioned priors online from the unlabeled test stream via lightweight second-order co-occurrence statistics, adding negligible overhead beyond a single forward pass. Across standard multi-label benchmarks and multiple CLIP backbones, BCP consistently outperforms strong TTA baselines, e.g., improving RN50 average mAP from 57.31 to 69.22 and ViT-B/16 from 62.61 to 71.79.

多标签识别测试时适应贝叶斯推理CLIP

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。