用可学习的超图捕捉骨骼动作中多关节语义关系,提升识别准确率。
Adaptive Hyper-Graph Convolution Network for Skeleton-based Human Action Recognition with Virtual Connections

- 设计自适应超图结构,动态学习关节间的高阶关联
- 在NTU-60/120和NW-UCLA数据集上达到领先精度
- 引入虚拟连接增强语义信息,适合动作识别研究者
人体骨骼共享拓扑结构启发了图卷积网络(GCN)在动作识别中的应用。然而,现有大多数GCN依赖于相邻关节间由边(骨骼)构成的二元连接,忽略了构建多顶点卷积结构的潜力。尽管部分研究尝试使用超图表示拓扑,但其构造策略固定,限制了对动作内在复杂关系的自适应挖掘。本文提出自适应超图卷积网络(Hyper-GCN),通过训练过程中动态优化超图,揭示以动作驱动的多关节关联。此外,虚拟连接被引入以支持高效特征聚合,隐式扩展骨架内部依赖范围。通过将虚拟连接注入超图,不同动作类别的语义线索得以凸显。在NTU-60、NTU-120和NW-UCLA数据集上的实验表明,该方法优于当前最先进方法。代码已公开于https://github.com/6UOOON9/Hyper-GCN。
原文摘要 · Abstract (English)
The shared topology of human skeletons motivated the recent investigation of graph convolutional network (GCN) solutions for action recognition. However, most of the existing GCNs rely on the binary connection of two neighboring vertices (joints) formed by an edge (bone), overlooking the potential of constructing multi-vertex convolution structures. Although some studies have attempted to utilize hyper-graphs to represent the topology, they rely on a fixed construction strategy, which limits their adaptivity in uncovering the intricate latent relationships within the action. In this paper, we address this oversight and explore the merits of an adaptive hyper-graph convolutional network (Hyper-GCN) to achieve the aggregation of rich semantic information conveyed by skeleton vertices. In particular, our Hyper-GCN adaptively optimises the hyper-graphs during training, revealing the action-driven multi-vertex relations. Besides, virtual connections are often designed to support efficient feature aggregation, implicitly extending the spectrum of dependencies within the skeleton. By injecting virtual connections into hyper-graphs, the semantic clues of diverse action categories can be highlighted. The results of experiments conducted on the NTU-60, NTU-120, and NW-UCLA datasets demonstrate the merits of our Hyper-GCN, compared to the state-of-the-art methods. The code is available at https://github.com/6UOOON9/Hyper-GCN.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。