用自回归与自适应超图结合Transformer，提升骨骼动作识别准确率。
Autoregressive Adaptive Hypergraph Transformer for Skeleton-based Activity Recognition
- 自回归生成超边，学习骨架序列的长程依赖关系。
- 在三个数据集上优于现有超图模型，最高提升3.2%准确率。
- 适合做骨骼动作识别、人体行为分析的研究者参考。
仅用图卷积网络提取骨骼序列的多尺度上下文信息和高阶关联关系，难以实现有效的动作分类。超图卷积虽能缓解此问题，但无法捕捉长距离依赖。而Transformer在建模长距离依赖和复杂上下文特征方面表现优异。为此，我们提出自回归自适应超图变换器（AutoregAd-HGformer），实现相位内（自回归、离散）与相位外（自适应）的超图生成。相位内超图通过向量量化与强大的自回归先验，生成更具鲁棒性和信息量的表示，用于超边构建；相位外超图生成器提供一种无需模型依赖的超边学习方法，实现属性与输入骨骼嵌入对齐。该模型采用混合（监督与无监督）学习策略，探索了空间、时间与通道维度上的动作相关特征。大量实验与消融研究验证了该模型在NTU RGB+D、NTU RGB+D 120和NW-UCLA数据集上优于当前最优超图架构。
原文摘要 · Abstract (English)
Extracting multiscale contextual information and higher-order correlations among skeleton sequences using Graph Convolutional Networks (GCNs) alone is inadequate for effective action classification. Hypergraph convolution addresses the above issues but cannot harness the long-range dependencies. The transformer proves to be effective in capturing these dependencies and making complex contextual features accessible. We propose an Autoregressive Adaptive HyperGraph Transformer (AutoregAd-HGformer) model for in-phase (autoregressive and discrete) and out-phase (adaptive) hypergraph generation. The vector quantized in-phase hypergraph equipped with powerful autoregressive learned priors produces a more robust and informative representation suitable for hyperedge formation. The out-phase hypergraph generator provides a model-agnostic hyperedge learning technique to align the attributes with input skeleton embedding. The hybrid (supervised and unsupervised) learning in AutoregAd-HGformer explores the action-dependent feature along spatial, temporal, and channel dimensions. The extensive experimental results and ablation study indicate the superiority of our model over state-of-the-art hypergraph architectures on the NTU RGB+D, NTU RGB+D 120, and NW-UCLA datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。