让不懂编程的医生也能用自然语言分析运动数据。
BiomechAgent: AI-Assisted Biomechanical Analysis Through Code-Generating Agents
- 用自然语言生成代码,自动完成数据查询与可视化。
- 在步态分析等任务中准确率超70%,临床推理能力初现。
- 专有指令和工具集成显著提升复杂分析效果,适合临床研究者使用。
无标记运动捕捉正使定量运动分析日益普及,但数据分析对无编程背景的临床医生仍具挑战。我们提出BiomechAgent,一个通过自然语言实现生物力学分析的代码生成AI代理,支持用户查询数据库、生成可视化图表,甚至解释数据而无需编写代码。为评估其能力,我们构建了一个涵盖数据检索、可视化、活动分类、时间分割和临床推理的系统性基准。BiomechAgent在数据检索和可视化任务中表现稳健,且展现出初步的临床推理能力。通过自建数据集系统评估多个设计选择发现:融合生物力学知识的领域专用指令显著优于通用提示;集成经验证的步态事件检测专用工具,在复杂时空分析中大幅提升准确率,而基础代理在此类任务中表现不佳。此外,使用本地开源小模型替代前沿云大模型后,除数据库检索外,其他任务性能显著下降。总之,BiomechAgent使可访问的运动捕捉数据变得更实用、更易用。
原文摘要 · Abstract (English)
Markerless motion capture is making quantitative movement analysis increasingly accessible, yet analyzing the resulting data remains a barrier for clinicians without programming expertise. We present BiomechAgent, a code-generating AI agent that enables biomechanical analysis through natural language and allows users to querying databases, generating visualizations, and even interpret data without requiring users to write code. To evaluate BiomechAgent's capabilities, we developed a systematic benchmark spanning data retrieval, visualization, activity classification, temporal segmentation, and clinical reasoning. BiomechAgent achieved robust accuracy on data retrieval and visualization tasks and demonstrated emerging clinical reasoning capabilities. We used our dataset to systematically evaluate several of our design decisions. Biomechanically-informed, domain-specific instructions significantly improved performance over generic prompts, and integrating validated specialized tools for gait event detection substantially boosted accuracy on challenging spatiotemporal analysis where the base agent struggled. We also tested BiomechAgent using a local open-weight model instead of a frontier cloud based LLM and found that perform was substantially diminished in most domains other than database retrieval. In short, BiomechAgent makes the data from accessible motion capture and much more useful and accessible to end users.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。