arXiv:2609.02039cs.CV2026-09

无需源数据,在测试时用少量参数自适应提升视觉模型对缺失模态的识别能力。

Test-Time Logit Prompting for Source-Free Missing Modality Adaptation

论文配图:Test-Time Logit Prompting for Source-Free Missing Modality Adaptation
图 1 · 摘自论文原文
  • 测试时通过优化逻辑提示,动态调整预测置信度。
  • 在多个基准上实现最高8%的性能提升,仅需数百参数。
  • 适合隐私敏感场景如医疗、个性化AI等无源数据环境。

视觉语言模型(VLMs)通过大规模图像-文本对实现了优异性能,但在实际部署中常面临模态缺失问题,导致性能显著下降。现有方法依赖源训练数据学习模态补偿策略,但当原始数据因隐私、存储或访问限制无法获取时(如临床应用、个性化AI服务),难以适用。本文提出一种轻量级无源测试时自适应框架Test-Time Logit Prompting(TLP),解决模态缺失引发的预测偏差。TLP通过不确定性感知的提示优化和模态完整一致性正则化,在不访问源数据的前提下,自适应调节预测置信度并保持语义一致性。在多个视觉语言基准上的实验表明,TLP在模态缺失场景下持续提升识别性能,最高达8%提升,仅需数百可调参数和少量测试时优化步骤。

原文摘要 · Abstract (English)

Vision-language models (VLMs) have achieved remarkable performance by leveraging complementary information from large-scale image-text pairs. However, missing-modality inputs are commonly encountered during real-world deployment, often leading to significant performance degradation. Existing methods primarily enhance model robustness by learning modality compensation strategies from source training data. However, their reliance on source training data makes them difficult to apply when original data are unavailable due to privacy, storage, or accessibility constraints, such as clinical applications and personalized AI services. This raises an important yet underexplored question: can VLMs be efficiently adapted at test time for visual recognition with missing modalities without accessing source training data? To this end, we propose Test-Time Logit Prompting (TLP), a lightweight source-free test-time adaptation framework for visual recognition with missing modalities. To address missing-induced prediction shifts, TLP optimizes logit prompts with uncertainty-aware adjustment and modality-complete consistency regularization, adaptively adjusting prediction confidence while preserving semantic consistency. Extensive experiments across diverse vision-language benchmarks demonstrate that TLP consistently enhances recognition performance under missing-modality scenarios, achieving up to 8\% improvements while requiring only hundreds of tunable parameters and a few test-time optimization steps.

视觉语言模型测试时自适应缺失模态无源学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。