通过自适应上下文聚合,提升动作片段边界定位精度
ContextDet: Temporal Action Detection with Adaptive Context Aggregation
- 设计金字塔式自适应上下文聚合结构,动态捕捉长程与局部特征
- 在6个基准上实现更优检测精度,推理速度更快
- 适合需要精准时序定位的视频理解任务
时序动作检测(TAD)因动作片段长度不一、边界模糊而具挑战性。现有方法对邻近上下文处理缺乏区分,导致边界预测不准。本文提出单阶段ContextDet框架,首次在TAD中引入大核卷积。模型采用金字塔式自适应上下文聚合(ACA)结构,捕获长距离上下文并增强动作可区分性。ACA每层包含两个新模块:上下文注意力模块(CAM)通过上下文门控块(CGB)识别关键上下文、促进多样性并保持完整性;长上下文模块(LCM)结合大、小核卷积,自适应融合远距离上下文与细粒度局部特征。通过在ACA金字塔中调整大核长度,实现轻量高效上下文聚合与动作判别。在MultiThumos、Charades、FineAction、EPIC-Kitchens 100、Thumos14和HACS共六个主流TAD数据集上进行大量实验,结果表明该模型在保持更低推理速度的同时,性能优于多种先进方法。
原文摘要 · Abstract (English)
Temporal action detection (TAD), which locates and recognizes action segments, remains a challenging task in video understanding due to variable segment lengths and ambiguous boundaries. Existing methods treat neighboring contexts of an action segment indiscriminately, leading to imprecise boundary predictions. We introduce a single-stage ContextDet framework, which makes use of large-kernel convolutions in TAD for the first time. Our model features a pyramid adaptive context aggragation (ACA) architecture, capturing long context and improving action discriminability. Each ACA level consists of two novel modules. The context attention module (CAM) identifies salient contextual information, encourages context diversity, and preserves context integrity through a context gating block (CGB). The long context module (LCM) makes use of a mixture of large- and small-kernel convolutions to adaptively gather long-range context and fine-grained local features. Additionally, by varying the length of these large kernels across the ACA pyramid, our model provides lightweight yet effective context aggregation and action discrimination. We conducted extensive experiments and compared our model with a number of advanced TAD methods on six challenging TAD benchmarks: MultiThumos, Charades, FineAction, EPIC-Kitchens 100, Thumos14, and HACS, demonstrating superior accuracy at reduced inference speed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。