提出轻量级动作识别系统，让嵌入设备实时处理视频
Real-Time Human Action Recognition on Embedded Platforms
- 设计单阶段神经网络IMFE，高效提取运动特征
- 在Jetson Xavier NX上实现30帧/秒实时识别
- 适合资源受限的边缘设备部署
随着计算机视觉和深度学习的发展，基于视频的人体动作识别（HAR）已具备可行性。然而，由于计算流程复杂，现有方法在嵌入式平台上处理实时视频流时延迟过高。本文通过四项贡献解决该问题：1）实验发现标准光流（OF）提取是当前先进HAR流水线中的主要延迟瓶颈；2）研究了传统与深度学习光流方法在延迟-准确率间的权衡，揭示了新型高效运动特征提取器的必要性；3）提出集成运动特征提取器（IMFE），一种单阶段神经网络架构，显著降低延迟；4）开发面向嵌入式平台的实时HAR系统RT-HARE。在Nvidia Jetson Xavier NX平台上的实验表明，RT-HARE可实现30帧/秒的实时动作识别，并保持高准确率。
原文摘要 · Abstract (English)
With advancements in computer vision and deep learning, video-based human action recognition (HAR) has become practical. However, due to the complexity of the computation pipeline, running HAR on live video streams incurs excessive delays on embedded platforms. This work tackles the real-time performance challenges of HAR with four contributions: 1) an experimental study identifying a standard Optical Flow (OF) extraction technique as the latency bottleneck in a state-of-the-art HAR pipeline, 2) an exploration of the latency-accuracy tradeoff between the standard and deep learning approaches to OF extraction, which highlights the need for a novel, efficient motion feature extractor, 3) the design of Integrated Motion Feature Extractor (IMFE), a novel single-shot neural network architecture for motion feature extraction with drastic improvement in latency, 4) the development of RT-HARE, a real-time HAR system tailored for embedded platforms. Experimental results on an Nvidia Jetson Xavier NX platform demonstrated that RT-HARE realizes real-time HAR at a video frame rate of 30 frames per second while delivering high levels of recognition accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。