轻量模型SV3.3B实现手机端高精度体育动作分析,关键帧提取+自监督学习。
SV3.3B: A Sports Video Understanding Model for Action Recognition
- 用时序运动差分采样与关键帧提取,精准捕捉动作起始、执行、收尾阶段。
- 在篮球数据集上比GPT-4o提升29.2%,信息密度和动作复杂度显著增强。
- 3.3B参数模型可本地部署,适合移动端体育分析与教练辅助场景。
本文针对自动化体育视频分析难题,提出轻量级3.3B参数模型SV3.3B。传统方法依赖服务器计算且难以捕捉精细生物力学变化,常遗漏准备、执行、收尾等关键秒级阶段。本工作结合新颖的时序运动差分采样与自监督学习,实现高效本地部署。采用基于DWT-VGG16-LDA的关键帧提取机制,从体育序列中智能选出16个代表性帧;后续使用预训练于掩码去噪目标的V-DWT-JEPA2编码器与微调后的语言模型解码器生成动作描述。在NSVA篮球数据子集上评估显示,该模型在文本生成指标与体育专用评价标准上均优于更大规模闭源模型(如GPT-4o),同时计算开销显著降低。其生成描述在真实值验证中比GPT-4o提升29.2%,信息密度、动作复杂度与测量精度均大幅改善,具备全面运动分析能力。
原文摘要 · Abstract (English)
This paper addresses the challenge of automated sports video analysis, which has traditionally been limited by computationally intensive models requiring server-side processing and lacking fine-grained understanding of athletic movements. Current approaches struggle to capture the nuanced biomechanical transitions essential for meaningful sports analysis, often missing critical phases like preparation, execution, and follow-through that occur within seconds. To address these limitations, we introduce SV3.3B, a lightweight 3.3B parameter video understanding model that combines novel temporal motion difference sampling with self-supervised learning for efficient on-device deployment. Our approach employs a DWT-VGG16-LDA based keyframe extraction mechanism that intelligently identifies the 16 most representative frames from sports sequences, followed by a V-DWT-JEPA2 encoder pretrained through mask-denoising objectives and an LLM decoder fine-tuned for sports action description generation. Evaluated on a subset of the NSVA basketball dataset, SV3.3B achieves superior performance across both traditional text generation metrics and sports-specific evaluation criteria, outperforming larger closed-source models including GPT-4o variants while maintaining significantly lower computational requirements. Our model demonstrates exceptional capability in generating technically detailed and analytically rich sports descriptions, achieving 29.2% improvement over GPT-4o in ground truth validation metrics, with substantial improvements in information density, action complexity, and measurement precision metrics essential for comprehensive athletic analysis. Model Available at https://huggingface.co/sportsvision/SV3.3B.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。