通过轻量门控与上下文机制提升动作定位精度
Introducing Gating and Context into Temporal Action Detection
- 引入门控机制筛选关键时序特征，增强局部感知
- 利用边界帧构建上下文，跨注意力提升全局理解
- 在THUMOS14和EPIC-KITCHEN 100上持续超越基线
时序动作检测（TAD）旨在定位未剪辑视频中的动作并进行分类，但因动作重叠和时长不一仍具挑战。近期研究发现，TAD性能更依赖于Transformer结构设计而非自注意力机制本身。基于此，我们提出一种轻量但高效的特征提取流程：首先，采用多尺度并行卷积的局部分支，捕捉细粒度与粗粒度时序特征，并引入门控机制选择最相关特征；其次，设计上下文分支，以边界帧作为键值对，通过交叉注意力分析其与中心帧的关系，从而捕捉时序依赖并增强上下文理解。在THUMOS14和EPIC-KITCHEN 100等挑战性数据集上的实验表明，该方法在基准模型及现有方法上均实现一致提升。
原文摘要 · Abstract (English)
Temporal Action Detection (TAD), the task of localizing and classifying actions in untrimmed video, remains challenging due to action overlaps and variable action durations. Recent findings suggest that TAD performance is dependent on the structural design of transformers rather than on the self-attention mechanism. Building on this insight, we propose a refined feature extraction process through lightweight, yet effective operations. First, we employ a local branch that employs parallel convolutions with varying window sizes to capture both fine-grained and coarse-grained temporal features. This branch incorporates a gating mechanism to select the most relevant features. Second, we introduce a context branch that uses boundary frames as key-value pairs to analyze their relationship with the central frame through cross-attention. The proposed method captures temporal dependencies and improves contextual understanding. Evaluations of the gating mechanism and context branch on challenging datasets (THUMOS14 and EPIC-KITCHEN 100) show a consistent improvement over the baseline and existing methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。