arXiv:2602.08726cs.CV2026-02中稿 · the 2nd Workshop o…被引 1

用Blender生成神经形态眼动数据,提升脉冲神经网络在真实场景的泛化能力

SynSacc: A Blender-to-V2E Pipeline for Synthetic Neuromorphic Eye-Movement Data and Sim-to-Real Spiking Model Training

  • 通过Blender模拟眼球快速移动与注视,生成可控的神经形态事件数据
  • 脉冲网络在真实数据上微调后达0.83准确率,且对时间分辨率变化保持稳定
  • 合成数据+脉冲网络显著降低计算开销,适合实时视觉系统开发

眼动研究,尤其是扫视与注视行为,是理解人类认知与感知机制的基础。精确分类需依赖能捕捉快速动态、无失真的传感技术。事件相机(又称动态视觉传感器,DVS)可异步记录光强变化,消除传统帧式相机的运动模糊,具备更高的时间分辨率和数据效率。本研究利用Blender构建合成数据集,模拟在受控条件下的人眼扫视与注视行为。基于脉冲神经网络(SNNs),我们评估了其鲁棒性,训练两种架构并在真实事件数据上进行微调。所提模型最高达到0.83准确率,在不同时间分辨率下表现一致,验证了分类稳定性。此外,使用合成事件流训练的SNN相较人工神经网络(ANN)显著提升计算效率,凸显合成数据增强在事件视觉中的价值。所有代码与数据已开源至https://github.com/Ikhadija-5/SynSacc-Dataset。

原文摘要 · Abstract (English)

The study of eye movements, particularly saccades and fixations, are fundamental to understanding the mechanisms of human cognition and perception. Accurate classification of these movements requires sensing technologies capable of capturing rapid dynamics without distortion. Event cameras, also known as Dynamic Vision Sensors (DVS), provide asynchronous recordings of changes in light intensity, thereby eliminating motion blur inherent in conventional frame-based cameras and offering superior temporal resolution and data efficiency. In this study, we introduce a synthetic dataset generated with Blender to simulate saccades and fixations under controlled conditions. Leveraging Spiking Neural Networks (SNNs), we evaluate its robustness by training two architectures and finetuning on real event data. The proposed models achieve up to 0.83 accuracy and maintain consistent performance across varying temporal resolutions, demonstrating stability in eye movement classification. Moreover, the use of SNNs with synthetic event streams yields substantial computational efficiency gains over artificial neural network (ANN) counterparts, underscoring the utility of synthetic data augmentation in advancing event-based vision. All code and datasets associated with this work is available at https: //github.com/Ikhadija-5/SynSacc-Dataset.

神经形态视觉眼动追踪脉冲网络合成数据

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。