融合第一人称视觉与可穿戴传感器,构建隐私友好型运动障碍评估新基准。
EgoInertia-MI: A Multimodal Egocentric Vision and IMU Benchmark for Motor Impairment Assessment

- 采集第一人称视频与IMU信号同步数据,模拟轻中重度运动障碍
- 多模态融合在严重程度估计上达0.78宏平均F1,动作识别达0.93
- 适合神经疾病评估、可穿戴健康监测等应用研究者参考
运动障碍(如震颤、运动迟缓、步态异常和姿势不稳)广泛存在于多种神经与运动系统疾病中。传统临床评估常为间歇性,难以捕捉运动行为的细微时间变化。尽管可穿戴IMU与第三人称视频在客观评估中展现出潜力,但后者存在隐私问题且需受控采集环境。相比之下,第一人称视觉提供了更自然、隐私友好的替代方案。本文提出EgoInertia-MI,一个融合同步第一人称视频与可穿戴IMU信号的多模态基准数据集,用于运动障碍分析。数据集包含19项上下肢活动,由健康志愿者模拟无障碍、轻度和重度三种障碍程度。我们设立两个基准任务:动作识别与运动障碍严重程度估计,并评估多种单模态与多模态基线模型。实验表明,第一人称视频提供强判别线索,而多模态融合表现最佳,严重程度估计达0.78宏平均F1,动作识别达0.93。结果凸显了结合第一人称视觉与可穿戴传感在生态有效且隐私友好的运动评估中的潜力。代码与数据见:https://fatemah-alh.github.io/EgoInertia-MI-Page/。
原文摘要 · Abstract (English)
Motor impairments, including tremor, bradykinesia, gait abnormalities, and postural instability, are common across many neurological and movement-related conditions. Conventional clinical assessments are often intermittent and may fail to capture subtle temporal variations in motor behavior. While wearable IMUs and third-person video have shown promise for objective motor assessment, third-person recordings raise privacy concerns and require constrained acquisition setups. In contrast, egocentric vision provides a more naturalistic and privacyaware alternative. In this work, we introduce EgoInertia-MI, a multimodal benchmark dataset combining synchronized egocentric video and wearable IMU signals for motor impairment analysis. The dataset contains 19 upper- and lower-body activities performed by healthy volunteers simulating varying levels of motor impairment severity levels: no impairment, mild impairment, and severe impairment. We establish two benchmark tasks: action recognition and motor impairment severity estimation, and evaluate multiple unimodal and multimodal baselines. Experimental results show that egocentric video provides strong cues for motor impairment assessment, while multimodal fusion achieves the best overall performance, reaching 0.78 Macro-F1 for severity estimation and 0.93 Macro-F1 for action recognition. These findings highlight the potential of combining egocentric vision and wearable sensing for ecologically valid and privacy-aware motor assessment. Code and data are available at:https://fatemah-alh.github.io/EgoInertia-MI-Page/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。