构建首个骨骼动作识别降级场景基准,揭示真实硬件下模型性能差异超40%。
SHARDeg: A Benchmark for Skeletal Human Action Recognition in Degraded Scenarios
- 在NTU-RGB+D-120数据集上建立三类真实降级场景的评估基准
- 发现降级类型导致模型准确率波动超过40%,帧时序不规则是关键因素
- 提出插值缓解方法提升现有模型性能,验证基于路径理论的鲁棒模型优势
针对实时与边缘计算中常见的视频流降级问题,本文为骨骼动作识别(SHAR)构建首个系统性降级基准。基于最大最细粒度的3D公开数据集NTU-RGB+D-120,评估五种主流SHAR模型在三种真实降级形式下的鲁棒性。结果表明,不同降级类型在等效帧率下可导致模型准确率相差>40%;分析发现帧时序不规则是性能差异主因。通过简单插值策略,可使现有模型性能提升>40%。此外,基于粗糙路径理论的LogSigRNN模型在低帧率(<10 FPS)下五项指标优于当前最优的DeGCN模型,平均高出6%,尽管其在高帧率(30 FPS)未达标,落后11-12%。
原文摘要 · Abstract (English)
Computer vision (CV) models for detection, prediction or classification tasks operate on video data-streams that are often degraded in the real world, due to deployment in real-time or on resource-constrained hardware. It is therefore critical that these models are robust to degraded data, but state of the art (SoTA) models are often insufficiently assessed with these real-world constraints in mind. This is exemplified by Skeletal Human Action Recognition (SHAR), which is critical in many CV pipelines operating in real-time and at the edge, but robustness to degraded data has previously only been shallowly and inconsistently assessed. Here we address this issue for SHAR by providing an important first data degradation benchmark on the most detailed and largest 3D open dataset, NTU-RGB+D-120, and assess the robustness of five leading SHAR models to three forms of degradation that represent real-world issues. We demonstrate the need for this benchmark by showing that the form of degradation, which has not previously been considered, has a large impact on model accuracy; at the same effective frame rate, model accuracy can vary by >40% depending on degradation type. We also identify that temporal regularity of frames in degraded SHAR data is likely a major driver of differences in model performance, and harness this to improve performance of existing models by up to >40%, through employing a simple mitigation approach based on interpolation. Finally, we highlight how our benchmark has helped identify an important degradation-resistant SHAR model based in Rough Path Theory; the LogSigRNN SHAR model outperforms the SoTA DeGCN model in five out of six cases at low frame rates by an average accuracy of 6%, despite trailing the SoTA model by 11-12% on un-degraded data at high frame rates (30 FPS).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。