构建首个统一标准的可穿戴动作识别基准,揭示性能已逼近天花板。
WHAR Arena: Benchmarking the State of the Art in Efficient Wearable Human Activity Recognition

- 30个数据集统一处理,17种模型在4760次训练中对比
- 性能趋稳,平均宏F1最高为CNN-HAR,但差距极小
- 小型模型如TinierHAR更适配设备部署,优于大型复杂模型
深度学习已成为可穿戴人体动作识别(WHAR)的主流范式,但因数据集、预处理和评估协议不一致,导致状态报告难以比较。本文构建一个大规模开源基准,整合30个多样化数据集,在标准化处理、统一模型接口和跨被试评估协议下,对17种代表性架构进行4760次训练评估。同时测量预测性能及在安卓参考设备上的延迟、峰值内存与模型大小。结果显示,当前最佳性能由多个模型共享,未被单一架构主导;尽管CNN-HAR达到最高平均宏F1,但顶尖模型表现紧密聚集,表明预测性能已接近天花板。考虑部署效率后,紧凑神经网络(如TinierHAR)和经典随机森林构成实际可行的帕累托前沿,而更大规模的循环或混合模型虽有高硬件开销却无性能提升。因此,尽管预测性能停滞,未来仍需聚焦部署效率优化与领域偏移适应能力提升。完整框架已公开,支持透明复用与扩展。
原文摘要 · Abstract (English)
Deep learning has become the dominant paradigm in Wearable Human Activity Recognition (WHAR), yet progress is obscured by a comparability crisis. Results are often reported using inconsistent datasets, custom data processing, and varying evaluation protocols, making state-of-the-art claims fragile. We address this with a large-scale, open-source benchmark that integrates 30 diverse datasets under standardized processing, unified model interfaces, and a shared cross-subject evaluation protocol. Evaluating 17 representative architectures across 4760 training runs, we jointly measure predictive performance alongside on-device latency, peak memory, and model size on an Android reference device. Our results reveal that the WHAR state of the art is distributed rather than dominated by a single architecture. While CNN-HAR achieves the highest mean macro-F1, top-performing models cluster tightly, indicating contemporary architectures have converged near a predictive performance ceiling. When accounting for deployment efficiency, compact neural models, such as TinierHAR, and classical Random Forests define the practically relevant Pareto frontier, whereas larger recurrent and hybrid models incur high hardware costs without corresponding performance gains. Consequently, while predictive performance has plateaued, substantial potential for future progress remains in optimizing deployment efficiency and improving adaptation to domain shifts. We release our full framework to support transparent reuse and extension.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。