arXiv:2504.01275eess.IVcs.SY2025-04被引 3

让图像传感器自己预测运动,超低功耗实时响应

A Retina-Inspired Pathway to Real-Time Motion Prediction inside Image Sensors for Extreme-Edge Intelligence

  • 模仿视网膜结构,在像素级实现运动预测
  • 实测每预测一次仅耗电18.56皮焦,延迟极低
  • 适合自动驾驶、机器人等边缘智能场景

实时运动预测对动物生存行为(如攻击与逃逸)至关重要,其机制起源于视网膜。类似地,计算机视觉系统若能在摄像头像素层直接实现运动预测,将极大提升自主系统的响应能力。为此,本文提出一种仿视网膜的类脑架构,可在全局晶圆22nm FDSOI工艺下实现像素级、低功耗、实时运动预测(MP)。该硬件-算法框架集成双相滤波器、脉冲累加器、非线性电路及二维多方向预测阵列,并通过3D Cu-Cu混合键合技术整合传感与计算芯片,显著减少面积占用并简化布线。在真实物体刺激下验证,系统实现了高效低延迟的预测计算,混合信号硬件实现中每帧预测仅消耗18.56皮焦能量。

原文摘要 · Abstract (English)

The ability to predict motion in real time is fundamental to many maneuvering activities in animals, particularly those critical for survival, such as attack and escape responses. Given its significance, it is no surprise that motion prediction in animals begins in the retina. Similarly, autonomous systems utilizing computer vision could greatly benefit from the capability to predict motion in real time. Therefore, for computer vision applications, motion prediction should be integrated directly at the camera pixel level. Towards that end, we present a retina-inspired neuromorphic framework capable of performing real-time, energy-efficient MP directly within camera pixels. Our hardware-algorithm framework, implemented using GlobalFoundries 22nm FDSOI technology, integrates key retinal MP compute blocks, including a biphasic filter, spike adder, nonlinear circuit, and a 2D array for multi-directional motion prediction. Additionally, integrating the sensor and MP compute die using a 3D Cu-Cu hybrid bonding approach improves design compactness by minimizing area usage and simplifying routing complexity. Validated on real-world object stimuli, the model delivers efficient, low-latency MP for decision-making scenarios reliant on predictive visual computation, while consuming only 18.56 pJ/MP in our mixed-signal hardware implementation.

运动预测类脑计算边缘智能低功耗

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。