用超图建模视觉拓扑关系,提升图像理解的上下文感知能力
HGFormer: Topology-Aware Vision Transformer with HyperGraph Learning
- 通过语义引导的超图构建,捕捉局部区域间的高阶关联
- 引入拓扑感知注意力机制,在信息聚合中保留空间结构
- 适用于需要精细场景理解的任务,如目标检测与分割
近年来,视觉变换器在计算机视觉领域得到广泛应用。尽管诸多工作引入了视觉特定的先验知识,但其隐含的排列不变性及令牌间全连接交互破坏了局部上下文与空间拓扑结构,不利于高层次建模。这违背了感知组织中强调局部组块与整体拓扑的原则。为此,本文提出基于超图的感知探索框架,设计了一种拓扑感知的视觉变换器——超图变换器(HGFormer)。首先,提出中心采样K近邻(CS-KNN)算法,用于在超图构建过程中实现语义引导;其次,设计拓扑感知超图注意力(HGA)机制,将超图拓扑作为感知线索,指导超图消息传递中的全局信息聚合。以HGFormer为视觉主干网络,可生成高效且统一的表示,实现清晰、细致的场景刻画。实验证明,所提HGFormer在多个视觉基准上表现优于或媲美最新SOTA方法。大量消融实验与可视化分析充分验证了方法的有效性与可解释性。
原文摘要 · Abstract (English)
The computer vision community has witnessed an extensive exploration of vision transformers in the past two years. Drawing inspiration from traditional schemes, numerous works focus on introducing vision-specific inductive biases. However, the implicit modeling of permutation invariance and fully-connected interaction with individual tokens disrupts the regional context and spatial topology, further hindering higher-order modeling. This deviates from the principle of perceptual organization that emphasizes the local groups and overall topology of visual elements. Thus, we introduce the concept of hypergraph for perceptual exploration. Specifically, we propose a topology-aware vision transformer called HyperGraph Transformer (HGFormer). Firstly, we present a Center Sampling K-Nearest Neighbors (CS-KNN) algorithm for semantic guidance during hypergraph construction. Secondly, we present a topology-aware HyperGraph Attention (HGA) mechanism that integrates hypergraph topology as perceptual indications to guide the aggregation of global and unbiased information during hypergraph messaging. Using HGFormer as visual backbone, we develop an effective and unitive representation, achieving distinct and detailed scene depictions. Empirical experiments show that the proposed HGFormer achieves competitive performance compared to the recent SoTA counterparts on various visual benchmarks. Extensive ablation and visualization studies provide comprehensive explanations of our ideas and contributions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。