轻量级骨架动作识别模型，用统一注意力减少计算开销
UniSTFormer: Unified Spatio-Temporal Lightweight Transformer for Efficient Skeleton-Based Action Recognition
- 将时空建模融合到单一注意力模块中，避免重复计算
- 参数量减少58%以上，计算成本降低60%以上，精度仍领先
- 适合资源受限场景的实时动作识别应用
基于骨架的动作识别（SAR）在使用Transformer架构后取得了显著进展。然而，现有方法通常依赖复杂的模块组合和沉重的设计，导致参数量大、计算成本高且可扩展性差。本文提出一种统一的时空轻量级Transformer框架，将空间与时间建模整合于单一注意力模块中，无需独立的时间建模模块。该设计减少了冗余计算，同时在空间建模过程中保持时间感知能力。此外，引入简化多尺度池化融合模块，结合局部与全局池化路径，增强对细微局部动作和整体运动模式的捕捉能力。在多个基准数据集上的大量实验表明，该轻量级模型在准确率与效率之间实现了更优平衡：相比当前最优的Transformer基线，参数复杂度降低超过58%，计算成本降低超过60%，同时保持具有竞争力的识别性能。
原文摘要 · Abstract (English)
Skeleton-based action recognition (SAR) has achieved impressive progress with transformer architectures. However, existing methods often rely on complex module compositions and heavy designs, leading to increased parameter counts, high computational costs, and limited scalability. In this paper, we propose a unified spatio-temporal lightweight transformer framework that integrates spatial and temporal modeling within a single attention module, eliminating the need for separate temporal modeling blocks. This approach reduces redundant computations while preserving temporal awareness within the spatial modeling process. Furthermore, we introduce a simplified multi-scale pooling fusion module that combines local and global pooling pathways to enhance the model's ability to capture fine-grained local movements and overarching global motion patterns. Extensive experiments on benchmark datasets demonstrate that our lightweight model achieves a superior balance between accuracy and efficiency, reducing parameter complexity by over 58% and lowering computational cost by over 60% compared to state-of-the-art transformer-based baselines, while maintaining competitive recognition performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。