arXiv:2512.18969cs.CVcs.AI2025-12

提升零样本组合识别准确率,让模型更好理解物体与状态的搭配关系。

Self-Attention with State-Object Weighted Combination for Compositional Zero Shot Learning

  • 引入自注意力机制优化物体和状态分类器
  • 在组合时考虑状态与物体的加权影响,提升合理性
  • 在三个数据集上均优于现有方法,最高提升2.1%

物体识别已广泛应用于各行业,但多数应用仅识别物体本身,未考虑其关联状态。同时识别物体与状态的能力仍较弱。传统方法将状态与物体合并为单一类别训练,但需覆盖所有组合,数据收集与训练成本高。组合零样本学习(CZSL)通过将状态与物体作为独立类别训练,可在无全量组合数据情况下识别新组合。当前最优方法KG-SP虽采用独立分类器并用语义模型评估组合合理性,但在状态与物体识别精度上仍有提升空间,且忽略组合过程中的权重差异。本文提出SASOW,在KG-SP基础上引入自注意力机制增强分类器性能,并在组合阶段引入状态与物体的加权机制,提升生成组合的合理性与准确性。在MIT-States、UT Zappos和C-GQA三个基准数据集上的实验表明,相比OW-CZSL和KG-SP,SASOW在未见组合识别准确率上分别提升2.1%、1.7%和0.4%。

原文摘要 · Abstract (English)

Object recognition has become prevalent across various industries. However, most existing applications are limited to identifying objects alone, without considering their associated states. The ability to recognize both the state and object simultaneously remains less common. One approach to address this is by treating state and object as a single category during training. However, this approach poses challenges in data collection and training since it requires comprehensive data for all possible combinations. Compositional Zero-shot Learning (CZSL) emerges as a viable solution by treating the state and object as distinct categories during training. CZSL facilitates the identification of novel compositions even in the absence of data for every conceivable combination. The current state-of-the-art method, KG-SP, addresses this issue by training distinct classifiers for states and objects, while leveraging a semantic model to evaluate the plausibility of composed compositions. However, KG-SP's accuracy in state and object recognition can be further improved, and it fails to consider the weighting of states and objects during composition. In this study, we propose SASOW, an enhancement of KG-SP that considers the weighting of states and objects while improving composition recognition accuracy. First, we introduce self-attention mechanisms into the classifiers for states and objects, leading to enhanced accuracy in recognizing both. Additionally, we incorporate the weighting of states and objects during composition to generate more reasonable and accurate compositions. Our validation process involves testing SASOW on three established benchmark datasets. Experimental outcomes affirm when compared against OW-CZSL approach, KG-SP, SASOW showcases improvements of 2.1%, 1.7%, and 0.4% in terms of accuracy for unseen compositions across the MIT-States, UT Zappos, and C-GQA datasets, respectively.

零样本学习组合识别自注意力

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。