提出新型动态视觉传感器,通过模拟视网膜机制提升中频信息感知能力。
FS-DVS: A Frequency-Selective Dynamic Visual Sensing Paradigm for Enhancing Information Completeness

- 在事件触发前加入可学习的空间滤波器,模仿视网膜神经节细胞聚合机制
- 学习到的滤波器自发形成中心-周围结构,显著增强中空间频率响应
- 无需提高灵敏度或后处理,实现抗噪且生物合理的信息补全
动态视觉传感器(DVS)通过异步报告像素级亮度变化,具备极高的时间分辨率和动态范围。然而,传统DVS采用独立像素触发机制,忽略了生物视网膜神经节细胞(RGCs)的空间整合特性,导致缺乏对比敏感度函数(CSF)及对中空间频率的敏感性,从而造成亚阈值信号丢失,信息不完整。为此,本文提出FS-DVS(频率选择性动态视觉传感器),在事件触发前引入可学习的空间滤波器,以模拟RGC的聚合机制。通过构建可微分的事件仿真框架,该滤波器可与下游任务端到端优化。研究发现,从初始δ函数出发,学习得到的滤波器会自发演化为中心-周围模式,突出中频成分,与人类CSF一致。在目标检测和动作识别任务中均取得显著性能提升,且不同任务下均收敛至类人CSF特征,表明该中频选择机制具有普适性。相比盲目提高传感器灵敏度或依赖后处理,本范式实现了高噪声鲁棒性的选择性信息增强,为下一代类脑传感器提供了可靠、生物合理的蓝图。
原文摘要 · Abstract (English)
Dynamic vision sensors (DVS) offer exceptional temporal resolution and dynamic range by asynchronously reporting pixel-level intensity changes. However, conventional DVS rely on a per-pixel independent triggering mechanism, ignoring the spatial integration performed by biological retinal ganglion cells (RGCs). Consequently, they lack the contrast sensitivity function (CSF) and its inherent sensitivity to mid-spatial frequencies, which inevitably leads to information incompleteness due to sub-threshold signal loss. To bridge this gap, we propose FS-DVS (Frequency-Selective Dynamic Vision Sensor), a novel paradigm that integrates a learnable spatial filter strictly preceding the event triggering process to mimic the RGC aggregation mechanism. By developing a differentiable event simulation framework, the spatial filter can be optimized end-to-end with downstream tasks. Our study reveals that starting from a delta function, the learned spatial filters spontaneously evolve into center-surround patterns that emphasize mid-frequency components, consistently aligning with human CSF. Beyond achieving substantial performance gains in object detection and action recognition, the consistent convergence to human-like CSF characteristics across different tasks underscores the universality of this mid-frequency selective mechanism. Compared to naively increasing sensor sensitivity or relying on post-processing, our paradigm achieves selective information enhancement with high noise resilience, providing a robust, biologically plausible blueprint for next-generation neuromorphic sensors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。