arXiv:2603.03815cs.CV2026-03

通过结构感知提示适配,让模型更好理解未见过的属性与物体组合。

Structure-aware Prompt Adaptation from Seen to Unseen for Open-Vocabulary Compositional Zero-Shot Learning

  • 利用语义相似概念在嵌入空间中的局部结构一致性设计损失函数
  • 在多个基准上显著提升对未见组合的识别准确率
  • 可无缝接入现有提示调优方法,适合零样本学习研究者

开放词汇组合零样本学习(OV-CZSL)的目标是在开放词汇场景下识别属性-物体组合,涵盖已见和未见的组合。近期提示调优方法在封闭设置(仅评估已见属性与物体组合)中表现出强泛化能力,但在开放设置下直接应用难以推广至未见属性、物体及其组合。本文实验发现,语义相关的属性或物体在嵌入空间中倾向于形成一致的局部结构。基于此,提出结构感知提示适配(SPA)方法:训练阶段引入结构一致性损失(SCL),保持已见概念的局部结构;推理阶段采用结构引导适配策略(SAS),将未见概念与语义相近的已见概念结构对齐。SPA为即插即用设计,可集成至现有CZSL提示调优方法。大量实验证明,SPA在保持闭集性能的同时,显著提升开放词汇表现。

原文摘要 · Abstract (English)

The goal of Open-Vocabulary Compositional Zero-Shot Learning (OV-CZSL) is to recognize attribute-object compositions in the open-vocabulary setting, where compositions of both seen and unseen attributes and objects are evaluated. Recently, prompt tuning methods have demonstrated strong generalization capabilities in the closed setting, where only compositions of seen attributes and objects are evaluated, i.e., Compositional Zero-Shot Learning (CZSL). However, directly applying these methods to OV-CZSL may not be sufficient to generalize to unseen attributes, objects and their compositions, as it is limited to seen attributes and objects. Normally, when faced with unseen concepts, humans adopt analogies with seen concepts that have the similar semantics thereby inferring their meaning (e.g., "wet" and "damp", "shirt" and "jacket"). In this paper, we experimentally show that the distribution of semantically related attributes or objects tends to form consistent local structures in the embedding space. Based on the above structures, we propose Structure-aware Prompt Adaptation (SPA) method, which enables models to generalize from seen to unseen attributes and objects. Specifically, in the training stage, we design a Structure-aware Consistency Loss (SCL) that encourages the local structure's consistency of seen attributes and objects in each iteration. In the inference stage, we devise a Structure-guided Adaptation Strategy (SAS) that adaptively aligns the structures of unseen attributes and objects with those of trained seen attributes and objects with similar semantics. Notably, SPA is a plug-and-play method that can be seamlessly integrated into existing CZSL prompt tuning methods. Extensive experiments on OV-CZSL benchmarks demonstrate that SPA achieves competitive closed-set performance while significantly improving open-vocabulary results.

零样本学习提示调优开放词汇结构感知

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。