融合事件与图像数据，实现80帧以上高速精准特征跟踪
BlinkTrack: Feature Tracking over 80 FPS via Events and Images
- 用可微分卡尔曼滤波器融合事件与图像信息，提升跟踪精度
- 在多模态数据下超过80 FPS，在预处理事件数据下达100 FPS
- 适用于高速运动场景，适合机器人视觉与实时系统开发者
事件相机因其高时间分辨率和异步变化捕捉能力，在复杂条件下的特征跟踪中备受关注。然而，事件相机缺乏传统相机提供的精细纹理信息，易导致跟踪误差累积。为此，我们提出一种新框架BlinkTrack，通过融合事件数据与灰度图像实现高频特征跟踪。该方法将传统卡尔曼滤波扩展为基于学习的框架，分别在事件分支与图像分支中使用可微分卡尔曼滤波器，有效解决异步数据下的数据关联与融合问题。我们还构建了新的合成与增强数据集以更好评估模型。实验表明，BlinkTrack在多模态数据下超过80 FPS，预处理事件数据下可达100 FPS，显著优于现有方法。代码与数据集已开源。
原文摘要 · Abstract (English)
Event cameras, known for their high temporal resolution and ability to capture asynchronous changes, have gained significant attention for their potential in feature tracking, especially in challenging conditions. However, event cameras lack the fine-grained texture information that conventional cameras provide, leading to error accumulation in tracking. To address this, we propose a novel framework, BlinkTrack, which integrates event data with grayscale images for high-frequency feature tracking. Our method extends the traditional Kalman filter into a learning-based framework, utilizing differentiable Kalman filters in both event and image branches. This approach improves single-modality tracking and effectively solves the data association and fusion from asynchronous event and image data. We also introduce new synthetic and augmented datasets to better evaluate our model. Experimental results indicate that BlinkTrack significantly outperforms existing methods, exceeding 80 FPS with multi-modality data and 100 FPS with preprocessed event data. Codes and dataset are available at https://github.com/ColieShen/BlinkTrack.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。