融合骨骼、动态与时间关系，提升动作识别准确率
ARN-LSTM: A Multi-Stream Fusion Model for Skeleton-based Action Recognition
- 三路并行提取骨骼、运动与时间特征，再融合预测
- 在NTU RGB+D数据集上优于现有方法，群体活动识别更优
- 适合需要精准动作理解的智能监控与人机交互场景
本文提出ARN-LSTM架构，一种用于基于骨骼的动作识别的多流融合模型，旨在同时捕捉动作序列中的空间运动与时间动态。传统方法通常只关注空间或时间特征，限制了对复杂人类行为的理解。所提模型通过三路结构整合关节、运动和时间信息：一路提取骨骼特征，一路捕捉动态时间特征，另一路采用时序分布式长短期记忆（TD-LSTM）结合注意力关系网络（ARN）建模时间关联。各路输出经全连接层融合后进行最终动作预测。在NTU RGB+D 60和NTU RGB+D 120数据集上的实验表明，该模型性能优于现有方法，尤其在群体活动识别任务中表现突出。
原文摘要 · Abstract (English)
This paper presents the ARN-LSTM architecture, a novel multi-stream action recognition model designed to address the challenge of simultaneously capturing spatial motion and temporal dynamics in action sequences. Traditional methods often focus solely on spatial or temporal features, limiting their ability to comprehend complex human activities fully. Our proposed model integrates joint, motion, and temporal information through a multi-stream fusion architecture. Specifically, it comprises a jointstream for extracting skeleton features, a temporal stream for capturing dynamic temporal features, and an ARN-LSTM block that utilizes Time-Distributed Long Short-Term Memory (TD-LSTM) layers followed by an Attention Relation Network (ARN) to model temporal relations. The outputs from these streams are fused in a fully connected layer to provide the final action prediction. Evaluations on the NTU RGB+D 60 and NTU RGB+D 120 datasets outperform the superior performance of our model, particularly in group activity recognition.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。