受视网膜神经节细胞启发，实现低功耗高效事件相机
Neural Ganglion Sensors: Learning Task-specific Event Cameras Inspired by the Neural Circuit of the Human Retina
- 模仿视网膜神经节细胞机制，学习任务特异的时空感知核
- 在视频插值与光流任务中性能优于传统事件相机
- 适合边缘设备等需实时低功耗视觉流的应用
受人眼神经元数据高效脉冲机制启发，事件相机通过异步、像素级的强度变化而非固定帧率图像，实现了高时间分辨率与低功耗、低带宽。然而，与人眼视网膜神经节细胞（RGCs）能整合感受野内多个感光细胞信号以提取时空特征不同，传统事件相机在决策是否生成事件时未利用局部空间上下文。此外，人眼包含约20种并行工作的不同类型的RGCs，各自对特定特征或条件敏感。受此生物设计启发，本文提出神经节传感器（Neural Ganglion Sensors），扩展传统事件相机，学习任务特异的时空视网膜核（即模拟RGC“事件”）。我们在视频插值与光流两个挑战性任务上评估该设计，结果表明，该生物启发式感知方式在提升性能的同时降低了整体事件带宽。研究结果凸显了受RGC启发的事件传感器在边缘设备及其他对高效、高分辨率视觉流有需求的低功耗实时应用中的前景。
原文摘要 · Abstract (English)
Inspired by the data-efficient spiking mechanism of neurons in the human eye, event cameras were created to achieve high temporal resolution with minimal power and bandwidth requirements by emitting asynchronous, per-pixel intensity changes rather than conventional fixed-frame rate images. Unlike retinal ganglion cells (RGCs) in the human eye, however, which integrate signals from multiple photoreceptors within a receptive field to extract spatio-temporal features, conventional event cameras do not leverage local spatial context when deciding which events to fire. Moreover, the eye contains around 20 different kinds of RGCs operating in parallel, each attuned to different features or conditions. Inspired by this biological design, we introduce Neural Ganglion Sensors, an extension of traditional event cameras that learns task-specific spatio-temporal retinal kernels (i.e., RGC "events"). We evaluate our design on two challenging tasks: video interpolation and optical flow. Our results demonstrate that our biologically inspired sensing improves performance relative to conventional event cameras while reducing overall event bandwidth. These findings highlight the promise of RGC-inspired event sensors for edge devices and other low-power, real-time applications requiring efficient, high-resolution visual streams.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。