arXiv:2412.10846cs.CVcs.HC2024-12被引 6

用视觉识别日常动作,帮康复医生理解患者在家的手部使用情况。

Detecting Activities of Daily Living in Egocentric Video to Contextualize Hand Use at Home in Outpatient Neurorehabilitation Settings

  • 聚焦患者接触的物体,而非动作姿态,识别日常生活活动。
  • 在16名患者数据上实现平均加权F1达0.78,所有参与者均超0.5。
  • 对动作差异不敏感,适合上肢功能障碍患者的居家康复场景。

可穿戴第一人称摄像头与机器学习有望为临床医生提供中风或脊髓损伤后患者居家手部使用情况的更细致理解。但需结合具体情境(如活动类型与物品交互)才能有效解读指标并指导治疗。本文证明,以物体为中心的方法——关注患者互动的物品而非运动方式——可在真实康复环境中有效识别日常生活活动(ADL)。我们在野外采集的复杂数据集上评估模型,包含16名手功能受损患者共2261分钟的第一人称视频。通过利用预训练的物体检测和手物交互模型,系统在不同损伤程度与环境间表现稳健,最佳模型达到0.78 ± 0.12的平均加权F1分数,在留一被试交叉验证下所有参与者F1均高于0.5。定性分析显示,该方法生成了具有临床意义的功能性物品使用信息,且对个体动作差异具有鲁棒性,特别适用于普遍存在上肢功能障碍的康复场景。

原文摘要 · Abstract (English)

Wearable egocentric cameras and machine learning have the potential to provide clinicians with a more nuanced understanding of patient hand use at home after stroke and spinal cord injury (SCI). However, they require detailed contextual information (i.e., activities and object interactions) to effectively interpret metrics and meaningfully guide therapy planning. We demonstrate that an object-centric approach, focusing on what objects patients interact with rather than how they move, can effectively recognize Activities of Daily Living (ADL) in real-world rehabilitation settings. We evaluated our models on a complex dataset collected in the wild comprising 2261 minutes of egocentric video from 16 participants with impaired hand function. By leveraging pre-trained object detection and hand-object interaction models, our system achieves robust performance across different impairment levels and environments, with our best model achieving a mean weighted F1-score of 0.78 +/- 0.12 and maintaining an F1-score > 0.5 for all participants using leave-one-subject-out cross validation. Through qualitative analysis, we observe that this approach generates clinically interpretable information about functional object use while being robust to patient-specific movement variations, making it particularly suitable for rehabilitation contexts with prevalent upper limb impairment.

日常活动识别康复医学第一人称视频手部功能

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。