arXiv:2605.02596cs.LG2026-05被引 1

首个融合运动、环境与声音的可穿戴活动识别数据集,助力复杂日常行为精准分析。

HARMES: A Multi-Modal Dataset for Wearable Human Activity Recognition with Motion, Environmental Sensing and Sound

论文配图:HARMES: A Multi-Modal Dataset for Wearable Human Activity Recognition with Motion, Environmental Sensing and Sound
图 1 · 摘自论文原文
  • 整合腕部IMU、环境传感器与音频三模态数据,提升活动识别精度。
  • 覆盖20人80小时数据,15类日常活动每人大约3小时标注数据。
  • 适用于研究多模态融合在动作识别中的互补作用,适合可穿戴设备开发者。

由于各传感模态存在固有优劣,基于可穿戴设备的多模态人体活动识别(HAR)日益重要,尤其在识别日常生活活动(ADLs)时,单一模态常因信号模糊导致误判。本文提出HARMES,一个结合腕部惯性测量单元(IMU)、大气环境传感器(湿度、温度、气压)和音频的多模态数据集。数据来自20名参与者在其家中执行家务活动,总时长超80小时,每人约3小时标注数据,涵盖15类ADL。据我们所知,HARMES是首个融合此三类传感器的数据集,规模约为此前最大腕部惯性-声学数据集的六倍。通过跨被试泛化与消融实验发现,模态贡献具有活动依赖性,对仅靠运动数据难以区分的活动尤为互补。数据集已开源至Zenodo,GitHub提供加载与训练示例代码。

原文摘要 · Abstract (English)

With each sensing modality exhibiting inherent strengths and limitations, multi-modal approaches for wearable Human Activity Recognition (HAR) are becoming increasingly relevant -- particularly for recognizing Activities of Daily Living (ADLs), where individual modalities often produce ambiguous signals for similar or complex activities. This work introduces HARMES, a multi-modal wearable dataset combining three wrist-recorded modalities: motion sensing via an Inertial Measurement Unit (IMU), atmospheric environmental sensors (humidity, temperature, and pressure), and audio. Collected from 20 participants performing household activities in their own homes, HARMES totals over 80 hours of recorded data, with approximately three hours of labeled activity data per participant across 15 ADL classes. To the best of our knowledge, HARMES is the first dataset to combine this particular sensor trio, and it is nearly six times larger than the previously largest wrist-inertial-acoustic HAR dataset. In an extensive benchmark, we evaluate cross-subject generalization and conduct an ablation study revealing that modality contributions are activity-dependent and can provide complementary value, particularly for activities that are ambiguous from motion data alone. HARMES is freely available at Zenodo, alongside example code for loading the dataset and training models on GitHub.

多模态可穿戴活动识别数据集

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。