arXiv:2412.01267cs.CV2024-12被引 7

边端实时动作识别新框架,低延迟低功耗下保持高精度。

EdgeOAR: Real-time Online Action Recognition On Edge Devices

  • 基于早期退出和特定任务特征增强,动态优化视频初期帧特征。
  • 在UCF-101上比最先进方法降低99.23%延迟和99.28%能耗。
  • 适合资源受限的边端设备,兼顾实时性与识别准确率。

本文针对在线动作识别(OAR)面临的挑战,提出一种专为边端设备设计的新框架EdgeOAR。OAR需在严格延迟约束下对视频流进行即时行为分析,是边端计算实现实时反馈的关键。现有方法多依赖完整视频片段处理,在需要立即识别的场景中表现不足。EdgeOAR引入面向早期退出的任务特异性特征增强模块(TFEM),包含轻量级子模块,从时空维度优化特征;设计迭代训练策略,使TFEM能从视频起始阶段学习有效特征。此外,还提出基于逆信息熵(IIE)和模态一致性(MC)的融合模块,提升特征融合质量并优化退出决策。该设计有效应对两大难题:在初始帧有限的在线视频流中建模鲁棒的时空动作表示,以及在资源受限设备上平衡精度与效率。实验表明,在UCF-101数据集上,EdgeOAR相较最先进方法降低99.23%延迟与99.28%能耗,并在边端设备上实现足够准确率。

原文摘要 · Abstract (English)

This paper addresses the challenges of Online Action Recognition (OAR), a framework that involves instantaneous analysis and classification of behaviors in video streams. OAR must operate under stringent latency constraints, making it an indispensable component for real-time feedback for edge computing. Existing methods, which typically rely on the processing of entire video clips, fall short in scenarios requiring immediate recognition. To address this, we designed EdgeOAR, a novel framework specifically designed for OAR on edge devices. EdgeOAR includes the Early Exit-oriented Task-specific Feature Enhancement Module (TFEM), which comprises lightweight submodules to optimize features in both temporal and spatial dimensions. We design an iterative training method to enable TFEM learning features from the beginning of the video. Additionally, EdgeOAR includes an Inverse Information Entropy (IIE) and Modality Consistency (MC)-driven fusion module to fuse features and make better exit decisions. This design overcomes the two main challenges: robust modeling of spatio-temporal action representations with limited initial frames in online video streams and balancing accuracy and efficiency on resource-constrained edge devices. Experiments show that on the UCF-101 dataset, our method EdgeOAR reduces latency by 99.23% and energy consumption by 99.28% compared to state-of-the-art (SOTA) method. And achieves an adequate accuracy on edge devices.

动作识别边端计算实时系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。