arXiv:2510.08305cs.CV2025-10中稿 · IEEE TCSVT被引 3

提出轻量长时程注意力机制,提升视频目标分割精度。

LTCA: Long-range Temporal Context Attention for Referring Video Object Segmentation

  • 用稀疏局部注意力与全局查询结合,平衡局部与全局信息
  • 在MeViS数据集上准确率提升11.3%和8.3%(验证集/测试集)
  • 适合需要高效处理长视频的视觉语言任务研究者

指称视频分割(RVOS)旨在根据语言描述分割视频中的目标。其关键在于从表达与视频的交互中提取长时程上下文信息,以刻画对象的动态属性。以往方法或对所有帧使用注意力,或堆叠密集局部注意力以获得全局视图,但难以平衡局部性与全局性,且计算复杂度随视频长度显著增加。本文提出一种高效的长时程时序上下文注意力(LTCA)机制,将全局上下文信息融入目标特征。具体而言,从两方面聚合全局信息:首先,通过跨帧的扩张窗口注意力堆叠多层,实现局部信息聚合并获得全局视角;其次,让每个查询随机从全局池中选取一小批键进行注意力,增强全局性。此外,引入全局查询与所有其他查询交互,直接编码全局上下文。实验表明,该方法在四个基准上达到新最优性能,尤其在MeViS val和test集上分别提升11.3%和8.3%。

原文摘要 · Abstract (English)

Referring Video Segmentation (RVOS) aims to segment objects in videos given linguistic expressions. The key to solving RVOS is to extract long-range temporal context information from the interactions of expressions and videos to depict the dynamic attributes of each object. Previous works either adopt attention across all the frames or stack dense local attention to achieve a global view of temporal context. However, they fail to strike a good balance between locality and globality, and the computation complexity significantly increases with the increase of video length. In this paper, we propose an effective long-range temporal context attention (LTCA) mechanism to aggregate global context information into object features. Specifically, we aggregate the global context information from two aspects. Firstly, we stack sparse local attentions to balance the locality and globality. We design a dilated window attention across frames to aggregate local context information and perform such attention in a stack of layers to enable a global view. Further, we enable each query to attend to a small group of keys randomly selected from a global pool to enhance the globality. Secondly, we design a global query to interact with all the other queries to directly encode the global context information. Experiments show our method achieves new state-of-the-art on four referring video segmentation benchmarks. Notably, our method shows an improvement of 11.3% and 8.3% on the MeViS valu and val datasets respectively.

视频分割注意力机制长时程建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。