arXiv:2504.18201cs.CVcs.AI2025-04被引 1

通过多粒度视觉线索组合,提升图像意图识别准确率与可解释性。

Multi-Grained Compositional Visual Clue Learning for Image Intent Recognition

  • 将意图识别分解为视觉线索的系统性组合,融合多粒度特征
  • 在Intentonomy和MDID数据集上达到当前最优性能
  • 适合研究人类表达理解、具身智能与可解释AI的学者

在社交媒体盛行的时代,人们频繁分享反映其意图与兴趣的图像,影响个人生活质量与社会稳定性。传统计算机视觉任务如目标检测和语义分割关注具体视觉表征,而意图识别更依赖隐含的视觉线索,面临线索多样性与主观性强、同一意图类别内部差异大的挑战。现有方法通过人工设计特征或基于全局特征构建类别原型,但仍难以应对各类意图的视觉多样性。本文提出一种名为多粒度组合视觉线索学习(MCCL)的新方法,借鉴人类认知的系统性组合特性,将意图识别拆解为视觉线索组合,并融合多粒度特征;采用类特定原型缓解数据不平衡问题;将意图识别建模为多标签分类,利用图卷积网络通过标签嵌入相关性注入先验知识。在Intentonomy和MDID数据集上实现当前最佳性能,同时具备良好可解释性。本工作为理解复杂多样的人类表达提供了新路径。

原文摘要 · Abstract (English)

In an era where social media platforms abound, individuals frequently share images that offer insights into their intents and interests, impacting individual life quality and societal stability. Traditional computer vision tasks, such as object detection and semantic segmentation, focus on concrete visual representations, while intent recognition relies more on implicit visual clues. This poses challenges due to the wide variation and subjectivity of such clues, compounded by the problem of intra-class variety in conveying abstract concepts, e.g. "enjoy life". Existing methods seek to solve the problem by manually designing representative features or building prototypes for each class from global features. However, these methods still struggle to deal with the large visual diversity of each intent category. In this paper, we introduce a novel approach named Multi-grained Compositional visual Clue Learning (MCCL) to address these challenges for image intent recognition. Our method leverages the systematic compositionality of human cognition by breaking down intent recognition into visual clue composition and integrating multi-grained features. We adopt class-specific prototypes to alleviate data imbalance. We treat intent recognition as a multi-label classification problem, using a graph convolutional network to infuse prior knowledge through label embedding correlations. Demonstrated by a state-of-the-art performance on the Intentonomy and MDID datasets, our approach advances the accuracy of existing methods while also possessing good interpretability. Our work provides an attempt for future explorations in understanding complex and miscellaneous forms of human expression.

意图识别多粒度特征可解释AI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。