用普通摄像头+大模型，低成本辅助肩关节疾病早期诊断。
Diagnosing Shoulder Disorders Using Multimodal Large Language Models and Consumer-Grade Cameras
- 分步用两个大模型分别处理动作识别与疾病诊断。
- 诊断准确率比直接视频分析提升79.6%。
- 适合医疗资源匮乏地区医生使用，可推广性强。
肩部疾病如冻结肩（又称粘连性关节囊炎）在全球范围内普遍存在，尤其在老年人和从事重复性肩部劳作的工作者中发病率高。在医疗资源稀缺地区，实现早期准确诊断面临巨大挑战，亟需低成本、易推广的辅助诊断方案。本研究提出利用消费级设备拍摄的视频进行诊断，降低用户成本。聚焦多模态大模型（MLLMs）在肩部疾病初步诊断中的创新应用，构建混合运动视频诊断框架（HMVDx）。该框架将动作理解与疾病诊断任务分离，由两个MLLM分别完成。除传统评估指标外，提出一种基于医疗决策流程（动作识别、运动诊断、最终诊断）的实用性指数（Usability Index），从全流程视角评估MLLM在医疗场景中的有效性，揭示低成本MLLM在医疗应用中的潜力。实验表明，与直接视频诊断相比，HMVDx在肩关节损伤诊断准确率上提升79.6%，为未来MLLM在医学视频理解中的应用提供了重要技术贡献。
原文摘要 · Abstract (English)
Shoulder disorders, such as frozen shoulder (a.k.a., adhesive capsulitis), are common conditions affecting the health of people worldwide, and have a high incidence rate among the elderly and workers engaged in repetitive shoulder tasks. In regions with scarce medical resources, achieving early and accurate diagnosis poses significant challenges, and there is an urgent need for low-cost and easily scalable auxiliary diagnostic solutions. This research introduces videos captured by consumer-grade devices as the basis for diagnosis, reducing the cost for users. We focus on the innovative application of Multimodal Large Language Models (MLLMs) in the preliminary diagnosis of shoulder disorders and propose a Hybrid Motion Video Diagnosis framework (HMVDx). This framework divides the two tasks of action understanding and disease diagnosis, which are respectively completed by two MLLMs. In addition to traditional evaluation indicators, this work proposes a novel metric called Usability Index by the logical process of medical decision-making (action recognition, movement diagnosis, and final diagnosis). This index evaluates the effectiveness of MLLMs in the medical field from the perspective of the entire medical diagnostic pathway, revealing the potential value of low-cost MLLMs in medical applications for medical practitioners. In experimental comparisons, the accuracy of HMVDx in diagnosing shoulder joint injuries has increased by 79.6\% compared with direct video diagnosis, a significant technical contribution to future research on the application of MLLMs for video understanding in the medical field.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。