提出双向自适应时序相关框架,提升事件相机光流估计精度与细节表现。
BAT: Learning Event-based Optical Flow with Bidirectional Adaptive Temporal Correlation
- 通过双向时序相关将稀疏事件转化为稠密空间运动特征
- 在DSEC-Flow上排名第一,显著优于现有方法,边缘更清晰
- 仅用历史事件即可预测未来光流,适合实时动态场景应用
事件相机具有高动态范围和高时间分辨率,对复杂光照和快速运动物体的光流估计有显著优势。当前主流事件相机光流方法多沿用图像基框架,但事件数据的空间稀疏性限制了性能。本文提出BAT框架,利用双向自适应时序相关估计事件基光流。创新设计包括:1)双向时序相关,将双向时序密集运动线索转换为空间稠密表示,实现精确且空间稠密的光流估计;2)自适应时序采样策略,保持相关性的时间一致性;3)空间自适应时序运动聚合,高效聚合一致目标运动特征并抑制不一致特征。实验结果在DSEC-Flow基准上排名第一,大幅超越现有最先进方法,同时呈现锐利边缘与高质量细节。值得注意的是,BAT仅依赖历史事件即可准确预测未来光流,显著优于E-RAFT的热启动方法。代码已开源。
原文摘要 · Abstract (English)
Event cameras deliver visual information characterized by a high dynamic range and high temporal resolution, offering significant advantages in estimating optical flow for complex lighting conditions and fast-moving objects. Current advanced optical flow methods for event cameras largely adopt established image-based frameworks. However, the spatial sparsity of event data limits their performance. In this paper, we present BAT, an innovative framework that estimates event-based optical flow using bidirectional adaptive temporal correlation. BAT includes three novel designs: 1) a bidirectional temporal correlation that transforms bidirectional temporally dense motion cues into spatially dense ones, enabling accurate and spatially dense optical flow estimation; 2) an adaptive temporal sampling strategy for maintaining temporal consistency in correlation; 3) spatially adaptive temporal motion aggregation to efficiently and adaptively aggregate consistent target motion features into adjacent motion features while suppressing inconsistent ones. Our results rank $1^{st}$ on the DSEC-Flow benchmark, outperforming existing state-of-the-art methods by a large margin while also exhibiting sharp edges and high-quality details. Notably, our BAT can accurately predict future optical flow using only past events, significantly outperforming E-RAFT's warm-start approach. Code: \textcolor{magenta}{https://github.com/gangweiX/BAT}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。