arXiv:2411.07945cs.CV2024-11被引 3

用简单结构实现视频定位新基准,效果媲美复杂模型。

SimBase: A Simple Baseline for Temporal Video Grounding

  • 用一维卷积替代复杂时序结构,简化时序建模
  • 仅用逐元素乘积实现跨模态交互,性能达顶尖水平
  • 适合想快速验证新想法的研究者参考

本文提出 SimBase,一个用于时序视频定位的简单但高效的基线方法。尽管近期研究在该任务上取得了显著进展,但网络架构趋向复杂,普遍依赖多种机制来捕捉时序关系和实现有效的多模态融合。与此相反,本文探讨:简化方法能达到怎样的效果?为此,设计了 SimBase,采用轻量级的一维时间卷积层替代复杂的时序结构,并仅使用逐元素乘积进行跨模态交互,避免繁琐的融合设计。令人惊讶的是,SimBase 在两个大规模数据集上均达到当前最优性能。作为一个简洁而强大的基线,我们希望 SimBase 能激发新思路,并推动未来时序视频定位研究的评估标准化。

原文摘要 · Abstract (English)

This paper presents SimBase, a simple yet effective baseline for temporal video grounding. While recent advances in temporal grounding have led to impressive performance, they have also driven network architectures toward greater complexity, with a range of methods to (1) capture temporal relationships and (2) achieve effective multimodal fusion. In contrast, this paper explores the question: How effective can a simplified approach be? To investigate, we design SimBase, a network that leverages lightweight, one-dimensional temporal convolutional layers instead of complex temporal structures. For cross-modal interaction, SimBase only employs an element-wise product instead of intricate multimodal fusion. Remarkably, SimBase achieves state-of-the-art results on two large-scale datasets. As a simple yet powerful baseline, we hope SimBase will spark new ideas and streamline future evaluations in temporal video grounding.

视频定位轻量模型多模态融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。