arXiv:2510.20162cs.CV2025-10NeurIPS

测试时动态积累图文知识,提升组合零样本识别效果

TOMCAT: Test-time Comprehensive Knowledge Accumulation for Compositional Zero-Shot Learning

  • 测试时从无监督数据中融合图文知识,更新多模态原型
  • 自适应权重控制原型调整程度,应对测试分布偏移
  • 用高置信图像队列保存历史视觉知识,适合新组合识别

组合零样本学习(CZSL)旨在基于已见属性-对象组合的知识,识别未见过的新组合。现有方法因测试时标签空间分布偏移而性能下降,根源在于属性与对象重新组合生成了未见组合。为此,我们提出一种新方法,在测试时从无监督数据中累积文本和视觉模态的全面知识,以更新多模态原型。在此基础上,设计自适应更新权重,灵活控制原型调整程度,使模型能适应测试时的分布偏移。此外,引入动态优先队列,存储高置信度图像,从历史图像中获取视觉知识用于推理。为保证多模态知识语义一致性,通过多模态协同表示学习对齐文本与视觉原型。大量实验表明,该方法在四个基准数据集上,于封闭世界与开放世界设置下均达到当前最优性能。代码将发布于 https://github.com/xud-yan/TOMCAT。

原文摘要 · Abstract (English)

Compositional Zero-Shot Learning (CZSL) aims to recognize novel attribute-object compositions based on the knowledge learned from seen ones. Existing methods suffer from performance degradation caused by the distribution shift of label space at test time, which stems from the inclusion of unseen compositions recombined from attributes and objects. To overcome the challenge, we propose a novel approach that accumulates comprehensive knowledge in both textual and visual modalities from unsupervised data to update multimodal prototypes at test time. Building on this, we further design an adaptive update weight to control the degree of prototype adjustment, enabling the model to flexibly adapt to distribution shift during testing. Moreover, a dynamic priority queue is introduced that stores high-confidence images to acquire visual knowledge from historical images for inference. Considering the semantic consistency of multimodal knowledge, we align textual and visual prototypes by multimodal collaborative representation learning. Extensive experiments indicate that our approach achieves state-of-the-art performance on four benchmark datasets under both closed-world and open-world settings. Code will be available at https://github.com/xud-yan/TOMCAT .

零样本学习图文对齐测试时学习组合识别

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。