让视觉Transformer动态调节记忆强度,提升少样本识别效果
Adaptive Hebbian Memory Routing in Vision Transformers for Few-Shot Learning

- 用轻量MLP路由控制记忆激活与更新强度
- 在CIFAR-FS上达96.94%准确率,优于固定记忆方法
- 适合需要快速适应新类别的少样本学习场景
少样本图像识别需模型从少量标注样本中快速适应新类别。传统赫布型快速记忆可提供临时关联信息,但固定行为未必适配所有任务。本文提出自适应赫布路由机制,通过轻量MLP路由控制记忆贡献、更新强度及历史记忆保留。在ViT-Small、DeiT-Small和Swin-Tiny上进行5路1样本评估,覆盖Omniglot、CIFAR-FS及跨域迁移(CIFAR-FS→Omniglot)。在直接对比中,自适应塑性将固定赫布结果从96.74%提升至96.92%,全自适应路由达最优96.94%。全自适应Swin模型推理时间由16.51ms降至14.05ms。多样本评估显示性能增益随支持样本数增加仍持续有效。结果表明,自适应塑性和记忆激活能超越固定赫布行为,显著提升少样本视觉变换器表征能力。
原文摘要 · Abstract (English)
Few-shot image recognition requires models to adapt to new classes from a small labeled support set. Hebbian fast-weight memory can provide temporary associative information during an episode, but fixed memory behavior may not be appropriate for every few-shot task. In this work, we propose Adaptive Hebbian Routing for few-shot Vision Transformers. The method uses a lightweight MLP router to control the contribution of Hebbian memory, the strength of memory updates, and the retention of previous memory from support-set features. We study Adaptive Placement, Adaptive Plasticity, and Fully Adaptive Hebbian Routing. Experiments use ViT-Small, DeiT-Small, and Swin-Tiny under 5-way 1-shot evaluation on Omniglot, CIFAR-FS, and cross-domain transfer from CIFAR-FS to Omniglot. In the direct Swin comparison, fixed and adaptive Hebbian variants use the same memory location. Adaptive Plasticity improves the fixed Hebbian result from 96.74\% to 96.92\%, while Fully Adaptive Routing achieves the best result at 96.94\%. The fully adaptive Swin model also reduces inference time from 16.51 ms to 14.05 ms relative to fixed Hebbian Swin. On CIFAR-FS, adaptive variants improve performance across all three backbones, and the multi-shot evaluation shows that these gains remain useful as the number of support examples increases. These results show that adaptive plasticity and adaptive memory activation can improve few-shot Transformer representations beyond fixed Hebbian behavior.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。