arXiv:2604.06614cs.CVcs.LG2026-04

针对部分标签场景,提出联合局部与全局信息的标签选择方法,提升提示学习鲁棒性。

Holistic Optimal Label Selection for Robust Prompt Learning under Partial Labels

论文配图:Holistic Optimal Label Selection for Robust Prompt Learning under Partial Labels
图 1 · 摘自论文原文
  • 结合邻居标签频率与softmax得分,从局部结构中筛选最可能标签。
  • 基于最优传输理论设计全局目标,优化批量内标签分布匹配度。
  • 在8个数据集上超越所有基线,适合弱监督场景下的提示学习应用。

提示学习作为参数高效的视觉-语言模型下游适配方法受到广泛关注。然而,在仅有部分标签的情况下,其性能常受限于标签模糊性和监督信息不足。为此,本文提出全盘最优标签选择(HopS),通过预训练特征编码器的泛化能力,采用两种互补策略:首先,设计基于局部密度的过滤器,从最近邻候选标签集中选取高频标签,并利用softmax得分识别最合理标签,捕捉特征空间中的结构规律;其次,引入基于最优传输的全局选择目标,将均匀采样分布映射到批量内候选标签分布,通过最小化期望运输成本确定最可能的标签分配。两种策略协同作用,实现从局部与全局视角的稳健标签选择。在8个基准数据集上的大量实验表明,HopS在部分监督下持续提升性能,优于所有基线。结果验证了全盘标签选择的有效性,为弱监督提示学习提供了实用解决方案。

原文摘要 · Abstract (English)

Prompt learning has gained significant attention as a parameter-efficient approach for adapting large pre-trained vision-language models to downstream tasks. However, when only partial labels are available, its performance is often limited by label ambiguity and insufficient supervisory information. To address this issue, we propose Holistic Optimal Label Selection (HopS), leveraging the generalization ability of pre-trained feature encoders through two complementary strategies. First, we design a local density-based filter that selects the top frequent labels from the nearest neighbors' candidate sets and uses the softmax scores to identify the most plausible label, capturing structural regularities in the feature space. Second, we introduce a global selection objective based on optimal transport that maps the uniform sampling distribution to the candidate label distributions across a batch. By minimizing the expected transport cost, it can determine the most likely label assignments. These two strategies work together to provide robust label selection from both local and global perspectives. Extensive experiments on eight benchmark datasets show that HopS consistently improves performance under partial supervision and outperforms all baselines. Those results highlight the merit of holistic label selection and offer a practical solution for prompt learning in weakly supervised settings.

提示学习弱监督标签选择

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。