用事件相机实现高效稀疏运动场与光流估计,速度提升30倍。
Learning Efficient Meshflow and Optical Flow from Event Cameras
- 设计轻量级EEMFlow网络,结合编码器-解码器结构快速估算运动场。
- 在1280x720高分辨率数据集上实现30倍于现有方法的推理速度。
- 提出自适应密度模块,提升模型在不同事件密度下的泛化性能。
本文研究基于事件相机的网格流(meshflow)估计问题,提出一种新型稀疏空间平滑运动场预测任务。为解决缺乏专用数据集和事件密度影响不足的问题,我们构建了大规模高分辨率事件网格流数据集HREM(1280x720),包含动态物体与复杂运动模式,并提供光流与网格流标签。同时提出轻量级EEMFlow网络,采用定制编码器-解码器架构,实现高效准确的网格流估计。进一步引入置信度驱动细节补全(CDC)模块,支持密集事件光流生成。实验表明,该模型相较最新方法提速30倍。扩展构建多密度数据集HREM+,并提出自适应密度模块(ADM),使EEMFlow与EEMFlow+性能分别提升8%和10%。代码与数据已开源。
原文摘要 · Abstract (English)
In this paper, we explore the problem of event-based meshflow estimation, a novel task that involves predicting a spatially smooth sparse motion field from event cameras. To start, we review the state-of-the-art in event-based flow estimation, highlighting two key areas for further research: i) the lack of meshflow-specific event datasets and methods, and ii) the underexplored challenge of event data density. First, we generate a large-scale High-Resolution Event Meshflow (HREM) dataset, which showcases its superiority by encompassing the merits of high resolution at 1280x720, handling dynamic objects and complex motion patterns, and offering both optical flow and meshflow labels. These aspects have not been fully explored in previous works. Besides, we propose Efficient Event-based MeshFlow (EEMFlow) network, a lightweight model featuring a specially crafted encoder-decoder architecture to facilitate swift and accurate meshflow estimation. Furthermore, we upgrade EEMFlow network to support dense event optical flow, in which a Confidence-induced Detail Completion (CDC) module is proposed to preserve sharp motion boundaries. We conduct comprehensive experiments to show the exceptional performance and runtime efficiency (30x faster) of our EEMFlow model compared to the recent state-of-the-art flow method. As an extension, we expand HREM into HREM+, a multi-density event dataset contributing to a thorough study of the robustness of existing methods across data with varying densities, and propose an Adaptive Density Module (ADM) to adjust the density of input event data to a more optimal range, enhancing the model's generalization ability. We empirically demonstrate that ADM helps to significantly improve the performance of EEMFlow and EEMFlow+ by 8% and 10%, respectively. Code and dataset are released at https://github.com/boomluo02/EEMFlowPlus.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。