CoachMe通过对比学习，让AI像教练一样精准指出运动错误并给出改进方法。
CoachMe: Decoding Sport Elements with a Reference-Based Coaching Instruction Generation Model
- 基于参考动作对比，从时序与物理层面识别动作差异
- 在花样滑冰和拳击上比GPT-4o高31.6%和58.3%的评分
- 适合体育教学、动作矫正与智能训练系统开发者
运动指导是帮助运动员通过分析动作并提供纠正建议来优化技术的重要任务。尽管多模态模型在动作理解方面取得进展，但生成精确且项目特定的指导仍具挑战，原因在于体育领域高度专业化以及对有效反馈的需求。我们提出CoachMe，一种基于参考的动作分析模型，通过在时间和物理维度上比较学习者动作与参考动作的差异，实现领域知识学习与类教练思维过程的构建，从而有效识别动作错误并提供改进说明。本文展示了CoachMe如何通过通用动作学习并利用少量数据适配花样滑冰和拳击等具体项目。实验表明，CoachMe生成的指导质量显著高于仅模仿教练语气而缺乏关键信息的指令。在花样滑冰上的G-Eval得分比GPT-4o高31.6%，拳击上高58.3%。分析进一步证实其能详细说明错误及其对应改进方法。
原文摘要 · Abstract (English)
Motion instruction is a crucial task that helps athletes refine their technique by analyzing movements and providing corrective guidance. Although recent advances in multimodal models have improved motion understanding, generating precise and sport-specific instruction remains challenging due to the highly domain-specific nature of sports and the need for informative guidance. We propose CoachMe, a reference-based model that analyzes the differences between a learner's motion and a reference under temporal and physical aspects. This approach enables both domain-knowledge learning and the acquisition of a coach-like thinking process that identifies movement errors effectively and provides feedback to explain how to improve. In this paper, we illustrate how CoachMe adapts well to specific sports such as skating and boxing by learning from general movements and then leveraging limited data. Experiments show that CoachMe provides high-quality instructions instead of directions merely in the tone of a coach but without critical information. CoachMe outperforms GPT-4o by 31.6% in G-Eval on figure skating and by 58.3% on boxing. Analysis further confirms that it elaborates on errors and their corresponding improvement methods in the generated instructions. You can find CoachMe here: https://motionxperts.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。