用脉冲神经网络学二进制编码,让跨模态检索更快更省电。
SpikeHash: Learning Binary Codes with Spiking Neural Networks for Cross-Modal Hashing Retrieval

- 把图文特征转为时间序列脉冲,统一建模哈希生成过程
- 在三个基准数据集上达到可比检索精度,参数量和能耗更低
- 适合追求低功耗、轻量化哈希模型的研究与应用
跨模态哈希检索将异构数据编码为紧凑的二进制码,以实现高效的汉明空间搜索。现有方法通常在连续特征空间中学习跨模态语义,并通过最终取符号操作生成二进制码,导致训练优化与离散哈希检索耦合较弱。我们提出SpikeHash,一种统一的脉冲神经网络框架,将跨模态哈希建模为脉冲状态演化、方向性脉冲交互和竞争性脉冲读出。具体而言,SpikeHash将图像和文本特征转化为多时间步脉冲序列,在共享汉明空间中,两个脉冲序列共同驱动共享哈希状态的时序演化。通过方向性脉冲调制进一步实现跨模态交互,使每种模态影响另一种的放电动态。关键的是,SpikeHash用正负脉冲通道间的时序竞争替代传统连续哈希头,每个哈希位由成对脉冲通道的竞争产生。实验结果表明,SpikeHash在三个基准数据集上实现了具有竞争力的检索准确率,同时显著降低了哈希学习阶段的参数量、运算量和估算能耗,为传统连续哈希映射提供了一种紧凑的脉冲替代方案。
原文摘要 · Abstract (English)
Cross-modal hashing retrieval encodes heterogeneous data into compact binary codes for efficient Hamming-space search. Existing methods usually learn cross-modal semantics in continuous feature spaces and generate binary codes through a final sign operation, which weakly couples training optimization with discrete hash retrieval. We propose SpikeHash, a unified spiking framework that formulates cross-modal hashing as spike-state evolution, directional spike interaction, and competitive spike readout. Specifically, SpikeHash converts image and text features into multi-timestep spike sequences. In a shared Hamming space, the two spike sequences jointly drive the temporal evolution of a shared hash state. Cross-modal interaction is further performed through directional spike modulation, enabling each modality to influence the firing dynamics of the other. Crucially, SpikeHash replaces the conventional continuous hash head with a positive-negative spiking hash readout, where each hash bit is produced by temporal competition between paired spike channels. Experimental results show that SpikeHash achieves competitive retrieval accuracy on three benchmark datasets while reducing the parameter size, operation count, and estimated energy of the hash learning stage, suggesting a compact spiking alternative to conventional continuous hash mapping. The project page is available at https://shuqiao-111.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。