用事件相机的法向流实现毫秒级运动分割,速度比现有方法快800倍。
Real-time Motion Segmentation with Event-based Normal Flow
- 以事件邻域法向流为输入,构建能量最小化模型求解分割
- 仅需少量候选运动模型即完成准确分割,计算复杂度显著降低
- 适用于实时动态场景理解,特别适合低延迟视觉任务
事件相机是仿生传感器,像素独立异步响应亮度变化,可在微秒级分辨率下工作,具备在极端条件下处理视觉任务的潜力。但由于单个事件信息稀疏,直接处理原始事件数据解决视觉任务效率极低,严重限制了先进方法在实时任务(如运动分割)中的应用。将法向流作为中间表示,从局部事件簇中压缩运动信息,可提供更有效的解决方案。本文提出一种基于法向流的事件视觉运动分割框架。利用直接从事件邻域学习到的稠密法向流作为输入,将运动分割建模为通过图割求解的能量最小化问题,并通过法向流聚类与运动模型拟合进行迭代优化。通过法向流驱动的运动模型初始化与拟合方法,系统仅需有限数量的候选模型即可高效估计独立运动物体的运动模型,大幅降低计算复杂度,实现实时性能,相比开源最先进方法提速近800倍。在多个公开数据集上的大量评估充分验证了该框架的准确性与高效性。
原文摘要 · Abstract (English)
Event-based cameras are bio-inspired sensors with pixels that independently and asynchronously respond to brightness changes at microsecond resolution, offering the potential to handle visual tasks in challenging scenarios. However, due to the sparse information content in individual events, directly processing the raw event data to solve vision tasks is highly inefficient, which severely limits the applicability of state-of-the-art methods in real-time tasks, such as motion segmentation, a fundamental task for dynamic scene understanding. Incorporating normal flow as an intermediate representation to compress motion information from event clusters within a localized region provides a more effective solution. In this work, we propose a normal flow-based motion segmentation framework for event-based vision. Leveraging the dense normal flow directly learned from event neighborhoods as input, we formulate the motion segmentation task as an energy minimization problem solved via graph cuts, and optimize it iteratively with normal flow clustering and motion model fitting. By using a normal flow-based motion model initialization and fitting method, the proposed system is able to efficiently estimate the motion models of independently moving objects with only a limited number of candidate models, which significantly reduces the computational complexity and ensures real-time performance, achieving nearly a 800x speedup in comparison to the open-source state-of-the-art method. Extensive evaluations on multiple public datasets fully demonstrate the accuracy and efficiency of our framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。