arXiv:2502.18982cs.CV2025-02被引 1

用事件流提升神经形态语义分割速度,兼顾低延迟与省电。

Enhanced Neuromorphic Semantic Segmentation Latency through Stream Event

  • 通过分析事件相机的事件流变化,动态调整处理策略
  • 在DSEC数据集上实现显著降延迟,精度损失小
  • 适合无人机、自动驾驶等低功耗实时场景

基于帧的视觉传感器在无人机和自动驾驶等实时系统中实现最优语义分割面临巨大挑战,传统方法难以平衡延迟、准确率与能效。本文利用事件相机(生物启发式传感器)产生的事件流,分析连续帧间的事件数量:高事件数表示场景变化大,低事件数表示变化小。通过引入脉冲神经网络(SNN),一种低功耗的类脑计算范式,有效利用事件信息完成语义分割任务。实验基于DSEC数据集表明,该方法显著降低延迟,仅带来有限精度下降;同时因采用SNN,实现极低功耗,适用于资源受限的实时应用。据我们所知,这是首个有效平衡低延迟、小精度损失与高能效的事件流增强语义分割方法。

原文摘要 · Abstract (English)

Achieving optimal semantic segmentation with frame-based vision sensors poses significant challenges for real-time systems like UAVs and self-driving cars, which require rapid and precise processing. Traditional frame-based methods often struggle to balance latency, accuracy, and energy efficiency. To address these challenges, we leverage event streams from event-based cameras-bio-inspired sensors that trigger events in response to changes in the scene. Specifically, we analyze the number of events triggered between successive frames, with a high number indicating significant changes and a low number indicating minimal changes. We exploit this event information to solve the semantic segmentation task by employing a Spiking Neural Network (SNN), a bio-inspired computing paradigm known for its low energy consumption. Our experiments on the DSEC dataset show that our approach significantly reduces latency with only a limited drop in accuracy. Additionally, by using SNNs, we achieve low power consumption, making our method suitable for energy-constrained real-time applications. To the best of our knowledge, our approach is the first to effectively balance reduced latency, minimal accuracy loss, and energy efficiency using events stream to enhance semantic segmentation in dynamic and resource-limited environments.

神经形态事件相机语义分割低功耗

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。