arXiv:2505.15325cs.CV2025-05IJCV被引 9

用可学习的软超边捕捉视觉特征的高阶关联,提升识别精度。

SoftHGNN: Soft Hypergraph Neural Networks for General Visual Recognition

  • 通过连续参与权重实现动态软超边建模,替代固定硬连接。
  • 在五个数据集上显著提升性能,最高增益达3.2%(如ImageNet)。
  • 轻量插件设计,适合集成到现有视觉模型中使用。

视觉识别依赖于理解图像标记的语义及其复杂交互。主流自注意力方法虽能建模全局成对关系,却难以捕捉真实场景中的高阶关联,且常伴随冗余计算。超图通过建模高阶交互,为解决此问题提供了新框架。然而,现有超图神经网络通常采用静态、硬性超边分配,导致冗余超边并忽略视觉语义的连续性。本文提出软超图神经网络(SoftHGNN),一种轻量级、可即插即用的后期语义推理方法,用于现有视觉流水线。SoftHGNN引入软超边概念:每个顶点通过连续可微的参与权重与超边关联,而非硬性二值分配。这些权重由顶点特征与少量可学习超边原型间的相似度生成,形成输入自适应且语义丰富的软超边。以软超边为信息聚合与传播媒介,SoftHGNN增强了特征表示的高阶上下文关联。为进一步提升扩展性,引入稀疏超边选择机制,仅激活前k个重要超边,并结合负载均衡正则项,确保超边利用充分且均衡。在五个数据集上的三项任务实验表明,SoftHGNN高效捕捉视觉场景中的高阶关联,实现显著性能提升。代码已开源:https://github.com/Mengqi-Lei/SoftHGNN。

原文摘要 · Abstract (English)

Visual recognition relies on understanding the semantics of image tokens and their complex interactions. Mainstream self-attention methods, while effective at modeling global pair-wise relations, fail to capture high-order associations inherent in real-world scenes and often suffer from redundant computation. Hypergraphs extend conventional graphs by modeling high-order interactions and offer a promising framework for addressing these limitations. However, existing hypergraph neural networks typically rely on static and hard hyperedge assignments, which lead to redundant hyperedges and overlooking the continuity of visual semantics. In this work, we present Soft Hypergraph Neural Networks (SoftHGNN), a lightweight plug-and-play hypergraph computation method for late-stage semantic reasoning in existing vision pipelines. Our SoftHGNN introduces the concept of soft hyperedges, where each vertex is associated with hyperedges via continuous and differentiable participation weights rather than hard binary assignments. These weights are produced by measuring similarities between vertex features and a small set of learnable hyperedge prototypes, yielding input-adaptive and semantically rich soft hyperedges. Using soft hyperedges as the medium for message aggregation and dissemination, SoftHGNN enriches feature representations with high-order contextual associations. To further enhance efficiency when scaling up the number of soft hyperedges, we incorporate a sparse hyperedge selection mechanism that activates only the top-k important hyperedges, along with a load-balancing regularizer to ensure adequate and balanced hyperedge utilization. Experimental results across three tasks on five datasets demonstrate that SoftHGNN efficiently captures high-order associations in visual scenes, achieving significant performance improvements. The code is available at: https://github.com/Mengqi-Lei/SoftHGNN.

超图神经网络视觉识别高阶关联轻量模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。