用Mamba提升视频哈希效率，兼顾速度与精度
Efficient Self-Supervised Video Hashing with Selective State Spaces
- 基于双向Mamba层，线性复杂度捕捉时序关系
- 自监督学习中引入中心对齐损失，加速收敛
- 适合需要高效视频检索的工程场景
自监督视频哈希（SSVH）在视频索引与检索中具有实际应用价值。尽管Transformer在时序建模上表现优异，但存在计算与内存效率低的问题。受先进状态空间模型Mamba启发，我们提出S5VH——一种基于Mamba的视频哈希模型，结合改进的自监督学习范式。设计双向Mamba层用于编码器与解码器，利用数据依赖的可选择扫描机制，在线性复杂度下有效捕捉时序关系。学习策略中，将特征空间中的全局语义转化为语义一致且判别性强的哈希中心，并引入中心对齐损失作为全局学习信号。提出的自监督-局部-全局（SLG）范式显著提升学习效率，实现更快更优的收敛。大量实验表明，S5VH在性能、迁移能力及推理效率方面均优于现有方法。代码已开源：https://github.com/gimpong/AAAI25-S5VH。
原文摘要 · Abstract (English)
Self-supervised video hashing (SSVH) is a practical task in video indexing and retrieval. Although Transformers are predominant in SSVH for their impressive temporal modeling capabilities, they often suffer from computational and memory inefficiencies. Drawing inspiration from Mamba, an advanced state-space model, we explore its potential in SSVH to achieve a better balance between efficacy and efficiency. We introduce S5VH, a Mamba-based video hashing model with an improved self-supervised learning paradigm. Specifically, we design bidirectional Mamba layers for both the encoder and decoder, which are effective and efficient in capturing temporal relationships thanks to the data-dependent selective scanning mechanism with linear complexity. In our learning strategy, we transform global semantics in the feature space into semantically consistent and discriminative hash centers, followed by a center alignment loss as a global learning signal. Our self-local-global (SLG) paradigm significantly improves learning efficiency, leading to faster and better convergence. Extensive experiments demonstrate S5VH's improvements over state-of-the-art methods, superior transferability, and scalable advantages in inference efficiency. Code is available at https://github.com/gimpong/AAAI25-S5VH.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。