arXiv:2506.23151cs.CVcs.AI2025-06ICCV被引 9

MEMFOF实现1080p高清光流估计算法,内存占用仅2.09GB

MEMFOF: High-Resolution Training for Memory-Efficient Multi-Frame Optical Flow Estimation

  • 采用低相关体积与高分辨率训练策略,降低内存开销
  • 1080p下训练仅需28.5GB显存,推理仅2.09GB显存
  • 在Spring、Sintel、KITTI等数据集上均达领先水平

近年来光流估计追求精度,导致GPU内存消耗持续增长,尤其在高分辨率(全高清)输入时更为显著。我们提出MEMFOF,一种内存高效的多帧光流估计方法,在多帧估计与显存占用之间取得良好平衡。特别地,该方法在1080p输入下运行时仅需2.09 GB GPU显存,训练时为28.5 GB,使其无需裁剪或降采样即可在原生1080p分辨率下进行训练。我们系统重审了类似RAFT架构的设计选择,结合缩减的相关体积、高分辨率训练协议与多帧估计,实现了多个基准上的最先进性能,同时大幅降低内存开销。本方法在准确率和运行效率上均优于更耗资源的替代方案,验证了其在高分辨率光流估计中的鲁棒性。提交时,该方法在Spring基准上以3.289%的1像素误差率排名第一,在Sintel(clean)上端点误差(EPE)为0.963,且在KITTI-2015 Fl-all指标上达到2.94%的最佳表现。代码已开源:https://github.com/msu-video-group/memfof。

原文摘要 · Abstract (English)

Recent advances in optical flow estimation have prioritized accuracy at the cost of growing GPU memory consumption, particularly for high-resolution (FullHD) inputs. We introduce MEMFOF, a memory-efficient multi-frame optical flow method that identifies a favorable trade-off between multi-frame estimation and GPU memory usage. Notably, MEMFOF requires only 2.09 GB of GPU memory at runtime for 1080p inputs, and 28.5 GB during training, which uniquely positions our method to be trained at native 1080p without the need for cropping or downsampling. We systematically revisit design choices from RAFT-like architectures, integrating reduced correlation volumes and high-resolution training protocols alongside multi-frame estimation, to achieve state-of-the-art performance across multiple benchmarks while substantially reducing memory overhead. Our method outperforms more resource-intensive alternatives in both accuracy and runtime efficiency, validating its robustness for flow estimation at high resolutions. At the time of submission, our method ranks first on the Spring benchmark with a 1-pixel (1px) outlier rate of 3.289, leads Sintel (clean) with an endpoint error (EPE) of 0.963, and achieves the best Fl-all error on KITTI-2015 at 2.94%. The code is available at https://github.com/msu-video-group/memfof.

光流估计内存优化高分辨率多帧

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。