arXiv:2503.18141cs.CV2025-03被引 4

用大模型分析走路姿态,自动评分还能给出医生级解释。

AGIR: Assessing 3D Gait Impairment with Reasoning based on LLMs

  • 先用自编码器提取动作特征,再让大模型学习动作与医学推理的对应关系。
  • 在帕金森病数据集上实现准确评分,相比现有方法提升12%以上。
  • 适合临床辅助诊断、可解释性要求高的医疗AI研究者使用。

步态评估对神经退行性疾病早期诊断、病情监测和治疗评价至关重要。尽管临床广泛应用,但存在主观性强、精度不足的问题。近年来深度学习虽提升了分类准确率,却缺乏可解释性,限制了其在临床决策中的应用。为此,我们提出AGIR:一个由预训练VQ-VAE动作标记器和微调后的大语言模型(LLM)组成的新型流程。为训练用于病理步态分析的LLM,我们构建了一个多模态数据集,在现有帕金森病步态数据基础上新增针对MDS-UPDRS步态评分的推理说明,并采用两阶段监督微调策略:1)生成阶段通过双向动作-描述生成对齐动作与分析描述;2)推理阶段引入逻辑链式思维(CoT)进行步态损伤评估与UPDRS评分。在现有数据集上的验证表明,该方法在准确性与鲁棒性上均优于当前最优模型,能从动作输入生成具有临床意义的评分与推理过程。

原文摘要 · Abstract (English)

Assessing gait impairment plays an important role in early diagnosis, disease monitoring, and treatment evaluation for neurodegenerative diseases. Despite its widespread use in clinical practice, it is limited by subjectivity and a lack of precision. While recent deep learning-based approaches have consistently improved classification accuracies, they often lack interpretability, hindering their utility in clinical decision-making. To overcome these challenges, we introduce AGIR, a novel pipeline consisting of a pre-trained VQ-VAE motion tokenizer and a subsequent Large Language Model (LLM) fine-tuned over pairs of motion tokens and Chain-of-Thought (CoT) reasonings. To fine-tune an LLM for pathological gait analysis, we first introduce a multimodal dataset by adding rationales dedicated to MDS-UPDRS gait score assessment to an existing PD gait dataset. We then introduce a two-stage supervised fine-tuning (SFT) strategy to enhance the LLM's motion comprehension with pathology-specific knowledge. This strategy includes: 1) a generative stage that aligns gait motions with analytic descriptions through bidirectional motion-description generation, 2) a reasoning stage that integrates logical Chain-of-Thought (CoT) reasoning for impairment assessment with UPDRS gait score. Validation on an existing dataset and comparisons with state-of-the-art methods confirm the robustness and accuracy of our pipeline, demonstrating its ability to assign gait impairment scores from motion input with clinically meaningful rationales.

步态分析大模型可解释性医疗AI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。