arXiv:2409.16953cs.CV2024-09NeurIPS被引 1

提出PASS框架,提升事件相机在不同频率下的识别能力。

PASS: Path-selective State Space Model for Event-based Recognition

  • 用状态空间模型自适应编码事件特征,按路径选择聚合
  • 在5个数据集上优于基线,频率变化时准确率下降仅8.62%
  • 适合需要高时间分辨率和跨频率泛化的事件识别任务

事件相机是生物启发式传感器,能以高时间分辨率异步捕捉亮度变化。现有事件识别方法通常在固定时间间隔采样并转换事件表示,受限于可处理的事件长度范围,且频率泛化能力差,未能充分利用其高时间分辨率优势。本文提出PASS框架,通过状态空间模型(SSMs)实现对事件的自适应编码,具备线性复杂度与频率泛化能力,更适用于高时间分辨率事件处理。我们设计了路径选择性事件聚合与扫描(PEAS)模块,通过自适应扫描与选择聚合事件表示,将特征编码为固定维度。同时引入多面选择引导(MSG)损失,降低编码过程中的随机性与冗余性。实验表明,PASS在五个公开数据集上均优于现有方法,在不同推理频率下表现更强泛化能力,准确率下降仅为8.62%(基线为20.69%)。整体上,PASS可建模长达1-10⁹数量级的事件长度,具备精准时间感知与真实场景下的良好泛化性能。

原文摘要 · Abstract (English)

Event cameras are bio-inspired sensors that capture intensity changes asynchronously with distinct advantages, such as high temporal resolution. Existing methods for event-based object/action recognition predominantly sample and convert event representation at every fixed temporal interval (or frequency). However, they are constrained to processing a limited number of event lengths and show poor frequency generalization, thus not fully leveraging the event's high temporal resolution. In this paper, we present our PASS framework, exhibiting superior capacity for spatiotemporal event modeling towards a larger number of event lengths and generalization across varying inference temporal frequencies. Our key insight is to learn adaptively encoded event features via the state space models (SSMs), whose linear complexity and generalization on input frequency make them ideal for processing high temporal resolution events. Specifically, we propose a Path-selective Event Aggregation and Scan (PEAS) module to encode events into features with fixed dimensions by adaptively scanning and selecting aggregated event presentations. On top of it, we introduce a novel Multi-faceted Selection Guiding (MSG) loss to minimize the randomness and redundancy of the encoded features during the PEAS selection process. Our method outperforms prior methods on five public datasets and shows strong generalization across varying inference frequencies with less accuracy drop (ours -8.62% vs. -20.69% for the baseline). Overall, PASS exhibits strong long spatiotemporal modeling for a broader distribution of event length (1-10^9), precise temporal perception, and generalization for real-world

事件相机状态空间模型时空建模泛化能力

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。