融合神经网络与符号优化,实现高效高精度光流估计。
NSFlow: End-to-End Differentiable Neuro-Symbolic Optical Flow for Visual Odometry

- 用CNN提取特征,通过可微分LK优化器端到端求解光流。
- 在动态光照和低纹理场景下比传统方法误差降低42%。
- 适合嵌入式实时视觉里程计系统,兼顾精度与速度。
稀疏光流在视觉里程计(VO)和视觉惯性里程计(VIO)中提供稳定的帧间对应关系。传统基于优化的方法(如Lucas-Kanade, LK)在小位移下表现良好,但对大运动和光照变化敏感。现代基于回归的深度学习方法虽更鲁棒,但计算开销大且缺乏显式几何一致性,难以用于高效VO/VIO前端。为此,我们提出一种混合神经符号框架,结合两者优势:使用卷积神经网络(CNN)提取鲁棒特征,输入可微分的LK优化器进行端到端光流估计。通过隐式微分,梯度可穿过迭代求解器,实现特征提取与光流估计的联合优化。该系统可无缝集成至现有VO/VIO流程,并在嵌入式平台实现实时运行。实验表明,在动态光照和低纹理等挑战场景下,本方法优于传统优化方法;相比纯回归方法,精度更高、延迟更低。部署于VIO系统时,平均误差降低42%,显著提升跟踪稳定性。代码已开源。
原文摘要 · Abstract (English)
Sparse optical flow provides stable inter-frame correspondence, playing a key role in Visual Odometry (VO) and Visual-Inertial Odometry (VIO). Classical optimization-based methods, such as Lucas-Kanade (LK), perform well under small displacements but are sensitive to large motions and illumination changes. Modern regression-based learning methods, while more robust in complex scenes, are often computationally heavy and lack explicit geometric consistency, making them less suitable for efficient VO/VIO front-ends. To bridge this gap, we propose a hybrid neuro-symbolic framework that combines the strengths of both paradigms. Our method uses a Convolutional Neural Network (CNN) to extract robust feature representations, which is fed into a differentiable LK optimizer to estimate optical flow in an end-to-end trainable manner. Through implicit differentiation, gradients are propagated across the iterative solver, enabling joint optimization of feature extraction and flow estimation. The resulting system integrates seamlessly into existing VO/VIO pipelines and runs in real-time on embedded platforms. Experiments show that our method outperforms conventional optimization-based flow in challenging conditions such as dynamic lighting and low texture, while also achieving higher accuracy and lower latency than purely regression-based alternatives. When deployed in a VIO system, our method demonstrates significant performance improvement, achieving an average error reduction of 42\% on challenging datasets while enhancing tracking stability. The code is publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。