受人眼视觉启发，动态选择关键图像区域以提升模型效率与鲁棒性。
Foveation-Guided Dynamic Token Selection for Robust and Efficient Vision Transformers
- 模仿人眼聚焦机制，动态筛选重要图像区域作为输入。
- 在50%计算预算下准确率81.9%，比DeiT-S高1个百分点，计算量减少34.57%。
- 适合对效率与抗干扰能力要求高的视觉任务，如边缘设备部署。
人类视觉系统（HVS）通过聚焦采样和眼球运动实现高效感知，节省代谢能量与计算资源。受此启发，我们提出福尔马特动态变压器（FDT），一种融合聚焦与视网膜周边视觉机制的视觉变换器架构。该模型在未显式训练对抗攻击或噪声的情况下仍表现出强鲁棒性。其核心由固定点模块（识别关键关注点以过滤无关信息）与视网膜周边模块（生成多尺度嵌入）构成。在50%固定点预算设置下，FDT准确率达81.9%，优于DeiT-S的80.9%，同时减少34.57%的乘加操作，展现出良好的精度-效率权衡。该设计为构建具备自适应计算与更强鲁棒性的类脑神经网络提供了新路径。
原文摘要 · Abstract (English)
The human visual system (HVS) employs foveated sampling and eye movements to achieve efficient perception, conserving both metabolic energy and computational resources. Drawing inspiration from this robustness and adaptability, we introduce the Foveated Dynamic Transformer (FDT), a foveation-guided dynamic token-selection architecture that integrates these mechanisms into a vision transformer framework. The FDT exhibits strong resilience to various types of noise and adversarial attacks, despite not being explicitly trained for such challenges. This inherent robustness is achieved through the use of fixation and foveation modules: the fixation module identifies fixation points to filter out irrelevant information, while the foveation module generates foveated embeddings with multi-scale information. At the 50% fixation-budget setting, FDT achieves higher accuracy than DeiT-S (81.9% vs. 80.9%) while reducing multiply-accumulate operations by 34.57%, highlighting one operating point on its accuracy-efficiency trade-off. These attributes position FDT as an HVS-inspired step toward artificial neural networks that combine adaptive computation with improved resilience.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。