用智能鞋垫+视频数据，低成本预测关节受力，助早期发现膝骨关节炎。
VidSole: A Multimodal Dataset for Joint Kinetics Quantification and Disease Detection with Deep Learning
- 用内置传感器鞋垫与多视角视频结合，采集足底受力与动作数据。
- 活动识别准确率达99.02%，膝内收力矩估计误差低于0.5%体质量×身高。
- 适合临床筛查、可穿戴设备研发及运动医学研究者使用。
理解关节内部负荷对诊断步态相关疾病（如膝骨关节炎）至关重要，但现有测量方法耗时、昂贵且仅限于实验室环境。本文通过三项关键贡献实现大规模、低成本的生物力学分析：开发并部署新型传感鞋垫，创建大型多模态生物力学数据集VidSole，以及构建基线深度学习管道以预测内部关节负荷。新型传感鞋垫可测量足底五个高压点的三轴力与力矩。VidSole包含超过2,600次试验中，52名不同参与者完成四种日常活动（起坐、坐起、行走、跑步）时，鞋垫测得的力与力矩、双视角RGB视频、3D人体运动捕捉数据及测力台数据。将鞋垫数据与从视频中提取的运动学参数（如姿态、膝角）输入深度学习模型，该模型由一个集成门控循环单元（GRU）活动分类器和针对各活动的独立长短期记忆（LSTM）回归网络构成，用于估计膝内收力矩（KAM），即膝骨关节炎的重要生物力学风险因子。活动分类准确率达99.02%，KAM估计均方绝对误差低于0.5%体质量×身高，达到当前临床检测膝骨关节炎的精度阈值，证明了本数据集在科研与临床中的实用性。
原文摘要 · Abstract (English)
Understanding internal joint loading is critical for diagnosing gait-related diseases such as knee osteoarthritis; however, current methods of measuring joint risk factors are time-consuming, expensive, and restricted to lab settings. In this paper, we enable the large-scale, cost-effective biomechanical analysis of joint loading via three key contributions: the development and deployment of novel instrumented insoles, the creation of a large multimodal biomechanics dataset (VidSole), and a baseline deep learning pipeline to predict internal joint loading factors. Our novel instrumented insole measures the tri-axial forces and moments across five high-pressure points under the foot. VidSole consists of the forces and moments measured by these insoles along with corresponding RGB video from two viewpoints, 3D body motion capture, and force plate data for over 2,600 trials of 52 diverse participants performing four fundamental activities of daily living (sit-to-stand, stand-to-sit, walking, and running). We feed the insole data and kinematic parameters extractable from video (i.e., pose, knee angle) into a deep learning pipeline consisting of an ensemble Gated Recurrent Unit (GRU) activity classifier followed by activity-specific Long Short Term Memory (LSTM) regression networks to estimate knee adduction moment (KAM), a biomechanical risk factor for knee osteoarthritis. The successful classification of activities at an accuracy of 99.02 percent and KAM estimation with mean absolute error (MAE) less than 0.5 percent*body weight*height, the current threshold for accurately detecting knee osteoarthritis with KAM, illustrates the usefulness of our dataset for future research and clinical settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。