多帧融合提升单目场景流估计，更好处理遮挡问题
RAFT-MSF++: Temporal Geometry-Motion Feature Fusion for Self-Supervised Monocular Scene Flow
- 用时序特征融合机制联合估计深度与三维运动
- 在KITTI数据集上达到24.14%的SF-all指标，提升30.99%
- 适合做自动驾驶、机器人视觉中的动态环境感知
单目场景流估计旨在从图像序列中恢复密集3D运动，但现有方法大多仅限于双帧输入，限制了时序建模能力并降低对遮挡的鲁棒性。我们提出RAFT-MSF++，一种自监督多帧框架，通过递归融合时序特征，联合估计深度与场景流。核心是几何-运动特征（GMF），紧凑编码耦合的运动与几何线索，并迭代更新以实现有效的时序推理。为增强对遮挡的鲁棒性，引入相对位置注意力注入空间先验，并设计遮挡正则化模块，将可靠运动从可见区域传播至模糊区域。这些组件使GMF能在复杂遮挡下有效传递信息。大量实验表明，RAFT-MSF++在KITTI Scene Flow基准上取得24.14%的SF-all指标，较基线提升30.99%，且在遮挡区域表现更优。代码已开源：https://github.com/sunzunyi/RAFT-MSF-PlusPlus。
原文摘要 · Abstract (English)
Monocular scene flow estimation aims to recover dense 3D motion from image sequences, yet most existing methods are limited to two-frame inputs, restricting temporal modeling and robustness to occlusions. We propose RAFT-MSF++, a self-supervised multi-frame framework that recurrently fuses temporal features to jointly estimate depth and scene flow. Central to our approach is the Geometry-Motion Feature (GMF), which compactly encodes coupled motion and geometry cues and is iteratively updated for effective temporal reasoning. To ensure the robustness of this temporal fusion against occlusions, we incorporate relative positional attention to inject spatial priors and an occlusion regularization module to propagate reliable motion from visible regions. These components enable the GMF to effectively propagate information even in ambiguous areas. Extensive experiments show that RAFT-MSF++ achieves 24.14% SF-all on the KITTI Scene Flow benchmark, with a 30.99% improvement over the baseline and better robustness in occluded regions. The code is available at https://github.com/sunzunyi/RAFT-MSF-PlusPlus.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。