提出结构化上下文学习，提升视频事件边界检测精度与效率
Structured Context Learning for Generic Event Boundary Detection
- 用结构化序列分割（SPoS）构建时序上下文，灵活适配各类模型
- 在Kinetics-GEBD等数据集上优于现有方法，实现更高准确率与更快速度
- 适合需要高效高精度视频理解的研究者和应用开发者
通用事件边界检测（GEBD）旨在识别人类感知的视频事件边界。本文提出一种新方法——结构化上下文学习，引入结构化序列分割（SPoS）为时序信息学习提供结构化上下文。该方法端到端可训练且不依赖特定时序模型（如GRU、LSTM、Transformer），具有更好速度-精度权衡。具体地，将输入帧序列通过SPoS分组，并为后续时序模型提供结构化上下文；SPoS整体计算复杂度随视频长度呈线性增长。随后计算组间相似性以捕捉帧间差异，再利用轻量全卷积网络基于分组相似性图判定事件边界。为缓解标注模糊问题，采用高斯核预处理真实标签。方法在Kinetics-GEBD、TAPOS及镜头切换检测数据集上广泛评估，显著优于现有最先进方法。
原文摘要 · Abstract (English)
Generic Event Boundary Detection (GEBD) aims to identify moments in videos that humans perceive as event boundaries. This paper proposes a novel method for addressing this task, called Structured Context Learning, which introduces the Structured Partition of Sequence (SPoS) to provide a structured context for learning temporal information. Our approach is end-to-end trainable and flexible, not restricted to specific temporal models like GRU, LSTM, and Transformers. This flexibility enables our method to achieve a better speed-accuracy trade-off. Specifically, we apply SPoS to partition the input frame sequence and provide a structured context for the subsequent temporal model. Notably, SPoS's overall computational complexity is linear with respect to the video length. We next calculate group similarities to capture differences between frames, and a lightweight fully convolutional network is utilized to determine the event boundaries based on the grouped similarity maps. To remedy the ambiguities of boundary annotations, we adapt the Gaussian kernel to preprocess the ground-truth event boundaries. Our proposed method has been extensively evaluated on the challenging Kinetics-GEBD, TAPOS, and shot transition detection datasets, demonstrating its superiority over existing state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。