用多尺度视频字幕理解羽毛球战术执行过程,支持中断后恢复的动态描述。
Shot2Tactic-Caption: Multi-Scale Captioning of Badminton Videos for Tactical Understanding
- 双分支结构分别生成击球级与战术级字幕,融合时空注意力机制。
- 构建首个含5494个击球/544个战术字幕的羽毛球数据集,支持动态战术识别。
- 引入击球级提示引导机制,可捕捉战术中断与重启,适合赛事分析场景。
羽毛球战术理解不仅涉及单个动作的识别,还需捕捉战术在时间维度上的动态执行过程。本文提出Shot2Tactic-Caption框架,实现羽毛球视频的语义与时间多尺度字幕生成,可同时输出描述个体击球行为的击球级字幕和反映动作序列演进的战术级字幕。我们构建了首个羽毛球字幕数据集Shot2Tactic-Caption Dataset,包含5,494个击球级字幕和544个战术级字幕。该框架采用双分支设计,两分支均包含视觉编码器、时空Transformer编码器和基于Transformer的解码器。为支持战术字幕生成,额外引入战术单元检测器,用于识别有效战术单元、战术类型及状态(如中断、恢复)。针对战术字幕生成,进一步设计击球级提示引导机制,将预测的战术类型与状态作为提示嵌入解码器的交叉注意力中。该机制使系统不仅能描述成功执行的战术,还能捕捉临时中断后恢复的战术过程。实验表明,该框架在生成击球与战术字幕方面均具有效性。消融实验显示,基于ResNet50的时空编码器优于其他变体,且击球级提示结构显著提升战术字幕的连贯性与准确性。
原文摘要 · Abstract (English)
Tactical understanding in badminton involves interpreting not only individual actions but also how tactics are dynamically executed over time. In this paper, we propose \textbf{Shot2Tactic-Caption}, a novel framework for semantic and temporal multi-scale video captioning in badminton, capable of generating shot-level captions that describe individual actions and tactic-level captions that capture how these actions unfold over time within a tactical execution. We also introduce the Shot2Tactic-Caption Dataset, the first badminton captioning dataset containing 5,494 shot captions and 544 tactic captions. Shot2Tactic-Caption adopts a dual-branch design, with both branches including a visual encoder, a spatio-temporal Transformer encoder, and a Transformer-based decoder to generate shot and tactic captions. To support tactic captioning, we additionally introduce a Tactic Unit Detector that identifies valid tactic units, tactic types, and tactic states (e.g., Interrupt, Resume). For tactic captioning, we further incorporate a shot-wise prompt-guided mechanism, where the predicted tactic type and state are embedded as prompts and injected into the decoder via cross-attention. The shot-wise prompt-guided mechanism enables our system not only to describe successfully executed tactics but also to capture tactical executions that are temporarily interrupted and later resumed. Experimental results demonstrate the effectiveness of our framework in generating both shot and tactic captions. Ablation studies show that the ResNet50-based spatio-temporal encoder outperforms other variants, and that shot-wise prompt structuring leads to more coherent and accurate tactic captioning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。