arXiv:2508.21816cs.CVcs.AI2025-08中稿 · ICDM 2025

挑战视觉事件识别中的模糊性,提出单正多标签学习新范式。

The Demon is in Ambiguity: Revisiting Situation Recognition with Single Positive Multi-Label Learning

  • 将事件分类从单标签改为单正多标签,捕捉语义重叠的复杂性
  • 在真实数据集上实现超过3%的MAP提升,同时保持传统精度
  • 设计新评估基准,适合多标签场景下的模型公平比较

情境识别(SR)是计算机视觉的基础任务,旨在通过识别关键事件及其关联实体,从图像中提取结构化语义摘要。具体包括:对输入图像进行主事件分类(动词分类)、识别参与实体及其语义角色(语义角色标注),并定位这些实体(语义角色定位)。现有方法将动词分类视为单标签问题,但本文通过全面分析发现,由于动词类别间普遍存在语义重叠,该设定无法处理视觉事件识别中的固有模糊性。本文提出三项贡献:首先,实证表明动词分类本质上是多标签问题;其次,鉴于大规模数据全多标签标注不现实,提出将动词分类重构为单正多标签学习(SPMLL)的新范式;第三,构建了专门用于多标签场景的综合性评估基准。为应对SPMLL挑战,提出图增强动词多层感知机(GE-VerbMLP),结合图神经网络建模标签相关性与对抗训练优化决策边界。在真实数据集上的大量实验表明,该方法在保持传统top-1和top-5准确率竞争力的同时,实现了超过3%的MAP提升。

原文摘要 · Abstract (English)

Context recognition (SR) is a fundamental task in computer vision that aims to extract structured semantic summaries from images by identifying key events and their associated entities. Specifically, given an input image, the model must first classify the main visual events (verb classification), then identify the participating entities and their semantic roles (semantic role labeling), and finally localize these entities in the image (semantic role localization). Existing methods treat verb classification as a single-label problem, but we show through a comprehensive analysis that this formulation fails to address the inherent ambiguity in visual event recognition, as multiple verb categories may reasonably describe the same image. This paper makes three key contributions: First, we reveal through empirical analysis that verb classification is inherently a multi-label problem due to the ubiquitous semantic overlap between verb categories. Second, given the impracticality of fully annotating large-scale datasets with multiple labels, we propose to reformulate verb classification as a single positive multi-label learning (SPMLL) problem - a novel perspective in SR research. Third, we design a comprehensive multi-label evaluation benchmark for SR that is carefully designed to fairly evaluate model performance in a multi-label setting. To address the challenges of SPMLL, we futher develop the Graph Enhanced Verb Multilayer Perceptron (GE-VerbMLP), which combines graph neural networks to capture label correlations and adversarial training to optimize decision boundaries. Extensive experiments on real-world datasets show that our approach achieves more than 3\% MAP improvement while remaining competitive on traditional top-1 and top-5 accuracy metrics.

情境识别多标签学习视觉理解

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。