arXiv:2508.16291cs.CVcs.MM2025-08被引 3

用双流Mamba网络分步评估花样滑冰技术与艺术分,提升长视频分析效率

Learning Long-Range Action Representation by Two-Stream Mamba Pyramid Network for Figure Skating Assessment

  • 双流设计:视觉流专评技术分,音视频融合流专评艺术分
  • 多尺度Mamba金字塔有效定位不同长度动作并给出精准评分
  • 适配真实裁判标准,尤其适合处理超长比赛视频的细粒度评估

花样滑冰的技术元素分(TES)与节目构成分(PCS)评估分别依赖于动作精度与艺术表现力。现有方法存在三大问题:一是未区分视频与音频线索在两类评分中的作用;二是将整段比赛视为整体预测总分,忽视动作元素的独立评价;三是长视频难以高效捕捉远距离时序依赖。为此,我们提出两流Mamba金字塔网络,分别构建基于视觉特征的TES评估流与融合音视频特征的PCS评估流。在PCS流中,引入多层级融合机制,确保视频特征在技术分评估时不被干扰,并通过金字塔各层级融合视听信息增强艺术分估计。在TES流中,采用多尺度Mamba金字塔结构与专用评分头,有效解决不同时间尺度动作的定位与评分难题。得益于Mamba对长程依赖的高效建模能力及线性计算复杂度,该方法能高效处理长时间比赛视频。大量实验表明,本框架在FineFS基准上达到当前最优性能。源代码已开源。

原文摘要 · Abstract (English)

Technical Element Score (TES) and Program Component Score (PCS) evaluations in figure skating demand precise assessment of athletic actions and artistic interpretation, respectively. Existing methods face three major challenges. Firstly, video and audio cues are regarded as common features for both TES and PCS predictions in previous works without considering the prior evaluation criterion of figure skating. Secondly, action elements in competitions are separated in time, TES should be derived from each element's score, but existing methods try to give an overall TES prediction without evaluating each action element. Thirdly, lengthy competition videos make it difficult and inefficient to handle long-range contexts. To address these challenges, we propose a two-stream Mamba pyramid network that aligns with actual judging criteria to predict TES and PCS by separating visual-feature based TES evaluation stream from audio-visual-feature based PCS evaluation stream. In the PCS evaluation stream, we introduce a multi-level fusion mechanism to guarantee that video-based features remain unaffected when assessing TES, and enhance PCS estimation by fusing visual and auditory cues across each contextual level of the pyramid. In the TES evaluation stream, the multi-scale Mamba pyramid and TES head we proposed effectively address the challenges of localizing and evaluating action elements with various temporal scales and give score predictions. With Mamba's superior ability to capture long-range dependencies and its linear computational complexity, our method is ideal for handling lengthy figure skating videos. Comprehensive experimentation demonstrates that our framework attains state-of-the-art performance on the FineFS benchmark. Our source code is available at https://github.com/ycwfs/Figure-Skating-Action-Quality-Assessment.

动作评估长序列建模Mamba多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。