将神经形态事件数据直接转为向量，让Transformer高效处理实时视觉任务。
Event2Vec: Processing Neuromorphic Events Directly by Representations in Vector Space
- 借鉴词向量思想，将异步稀疏事件映射为向量表示
- 在极低事件数和低分辨率下仍保持高精度与低延迟
- 适合需要实时处理的嵌入式神经形态视觉系统
神经形态事件相机相比传统相机具有更高的时间分辨率、能效和动态范围。然而，其异步稀疏的数据格式给传统深度学习方法带来挑战。现有方法要么将事件稠密化为帧，牺牲了稀疏异步特性；要么采用不兼容GPU加速的非规则模型。受词向量模型启发，我们提出event2vec，一种使Transformer能直接处理事件的新表征。在DVS Gesture、ASL-DVS和DVS-Lip基准上验证了其有效性，结果显示event2vec参数效率极高，具备高吞吐量与低延迟，在事件数量极少或空间分辨率较低时仍可实现高精度。结果表明，稀疏异步事件数据可直接融入高性能Transformer架构，为实时神经形态视觉提供高效范式。代码已公开于https://github.com/Intelligent-Computing-Lab-Panda/event2vec。
原文摘要 · Abstract (English)
Neuromorphic event cameras possess superior temporal resolution, power efficiency, and dynamic range compared to traditional cameras. However, their asynchronous and sparse data format poses a significant challenge for conventional deep learning methods. Most existing methods either densify events into frames, sacrificing their sparse asynchronous nature, or use irregular models that are less compatible with GPU acceleration. Inspired by word-to-vector models, we propose event2vec, a novel representation that allows Transformers to process events directly. We demonstrate the effectiveness of event2vec on the DVS Gesture, ASL-DVS, and DVS-Lip benchmarks, showing that event2vec is remarkably parameter-efficient, features high throughput and low latency, and achieves high accuracy even with an extremely low number of events or low spatial resolutions. These results show that sparse asynchronous event data can be directly integrated into high-throughput Transformer architectures, offering an efficient paradigm for real-time neuromorphic vision. The code is provided at https://github.com/Intelligent-Computing-Lab-Panda/event2vec.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。