arXiv:2604.18284cs.CV2026-04

用类脑脉冲机制让视觉提示更抗噪,零开销部署

Spike-NVPT: Learning Robust Visual Prompts via Bio-Inspired Temporal Filtering and Discretization

论文配图:Spike-NVPT: Learning Robust Visual Prompts via Bio-Inspired Temporal Filtering and Discretization
图 1 · 摘自论文原文
  • 基于脉冲神经元设计时序滤波层,积累有效信号
  • 输出稀疏二值提示,噪声鲁棒性提升11.2%
  • 适合需要轻量抗噪的视觉模型微调场景

预训练视觉模型已在多个领域广泛应用。基于提示调优的方法成为参数高效的适配范式。尽管在标准基准上表现良好,但连续密集的提示容易对输入噪声敏感,因高容量提示会过拟合无关细节。为此,我们提出Spike-NVPT,一种抗噪声的视觉提示调优方法。具体地,设计基于脉冲神经元的信号滤波层,利用整合-放电(IF)机制在时间上累积任务相关信号并过滤瞬时噪声波动;随后的脉冲离散化单元将过滤后的信号转化为稀疏二值提示。该离散化起到强正则化作用,迫使模型决策边界锚定在最具区分性和鲁棒性的特征上。值得注意的是,生成的二值提示在部署阶段保持静态,确保推理时无额外计算开销。实验结果表明,Spike-NVPT在鲁棒性上达到最大11.2%的性能提升,同时在干净数据集上保持有竞争力的准确率。据我们所知,这是首次尝试将脉冲神经元用于传统人工神经网络(ANN)基视觉模型的微调。

原文摘要 · Abstract (English)

Pre-trained vision models have found widespread application across diverse domains. Prompt tuning-based methods have emerged as a parameter-efficient paradigm for adapting pre-trained vision models. While effective on standard benchmarks, the continuous and dense nature of learned prompts can lead to sensitivity against input noise, as the high-capacity prompts tend to overfit task-irrelevant details. To address this trade-off, we propose Spike-NVPT, a noise-robust visual prompt tuning method. Specifically, we design a Signal Filtering Layer based on spiking neurons, which uses the integrate-and-fire (IF) mechanism to accumulate task-relevant signals over time and filter transient noise fluctuations. A subsequent Spike Discretization Unit converts filtered signals into sparse binary prompts. This discretization acts as a strong regularizer, forcing the model to anchor decision boundaries on the most discriminative and robust features. Notably, the resulting binary prompts remain static during deployment, ensuring zero additional computational overhead during inference. Experimental results demonstrate that Spike-NVPT achieves superior robustness performance, with a maximum improvement of 11.2% over conventional methods, and retains competitive accuracy on clean datasets. To the best of our knowledge, this is the first attempt to leverage spiking neurons for fine-tuning traditional artificial neural network (ANN)-based visual models.

提示调优脉冲神经网络抗噪

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。