用Mamba模型提升事件相机3D特征追踪的效率与长期稳定性
E-TraMamba: A New Paradigm for Efficient Long-Term 3D Feature Tracking with Event Cameras

- 采用线性状态空间模型实现高效长时序建模
- 在严格精度下特征存活时间超基准2倍以上,轨迹误差更低
- 适合低延迟视觉里程计与实时机器人导航场景
事件相机3D追踪可实现低延迟、高帧率感知,但现有基于CNN和Transformer的追踪器难以在稀疏、噪声大的事件流中捕捉长程时空依赖,尤其在实时与高效约束下表现受限。为此,我们提出E-TraMamba,首个基于Mamba架构的事件数据3D特征追踪框架。该框架采用线性状态空间模型实现高效长程建模,并集成轻量级仿射变换预测器,在运动模糊和遮挡下保持稳定追踪。我们还设计了一种有效融合多尺度信息(局部时空块、相关图、位置嵌入)的机制,生成统一表征以实现稳定平滑的3D追踪。构建了大规模合成数据集EvD-PointOdyssey,通过单目渲染生成同步事件流、深度图与精确3D轨迹,用于训练与评估。大量实验表明,E-TraMamba达到当前最优性能,在严格精度阈值(如0.1 m)下特征存活时间超过基线2倍以上,追踪特征占比更高,均方根误差更低,是低延迟视觉里程计、实时SLAM与交互式机器人的有力候选方案。
原文摘要 · Abstract (English)
Event-based 3D tracking enables low-latency and high-speed perception, while existing CNN- and Transformer-based trackers struggle to capture long-range spatiotemporal dependencies in sparse, noisy event streams, especially under real-time and efficiency constraints. To address these challenges, we present E-TraMamba, the first Mamba-based framework for 3D feature tracking on event data. This new framework adopts a linear state-space model for efficient long-range modeling and integrates a lightweight affine-transform predictor to maintain stable tracking under motion blur and occlusion. We also design an effective scheme to fuse multi-scale cues -- local spatiotemporal patches, correlation maps, and positional embeddings -- into a unified representation that enables stable and smooth 3D tracking. We construct a large-scale synthetic dataset, named EvD-PointOdyssey, which is generated with monocular rendering and provides synchronized event streams, depth maps, and accurate 3D trajectories for training and evaluating event-based 3D tracking models. Extensive experiments on event-based benchmarks demonstrate that E-TraMamba achieves state-of-the-art performance, delivering over $2\times$ longer feature lifetimes under strict accuracy thresholds (e.g., 0.1 m), with higher tracked-feature ratios and lower RMSE than all baselines. These results make E-TraMamba a strong candidate for low-latency visual odometry, real-time SLAM, and interactive robotics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。