提出高效采样方法，让光流估计更快更省内存。
Efficient All-Pairs Correlation Volume Sampling for Optical Flow Estimation
- 设计新算法，同时降低内存和计算开销。
- 在高分辨率下推理速度提升63%，内存降低99%。
- 适合需要高速高精度光流的实时应用。
近期光流估计方法常通过密集全对相关体积进行局部代价采样，导致像素数的平方级计算与内存复杂度。尽管存在按需计算代价的内存高效实现，但实际运行速度显著变慢，多数方法因此采用降采样图像，丢失细粒度信息。为此，本文提出一种既高效又节省内存的全对相关体积采样算法，完全匹配RAFT定义的数学运算。该方法相比按需采样最高提速92%，内存使用相同；与默认实现相比，内存消耗降低至少99%，且性能不逊色。由于代价采样占整体运行时间较大比例，此改进可使高分辨率输入的端到端模型推理最多节省63%时间。评估包含8K超高清数据集及SEA-RAFT推理扩展，实现在高分辨率下准确率与运行效率双领先。
原文摘要 · Abstract (English)
Recent optical flow estimation methods often employ local cost sampling from a dense all-pairs correlation volume. This results in quadratic computational and memory complexity in the number of pixels. Although an alternative memory-efficient implementation with on-demand cost computation exists, this is significantly slower in practice and therefore many prior methods process images at downsampled resolutions, missing fine-grained details. To address this, we propose an algorithm for both memory and compute-efficient implementation of the all-pairs correlation volume sampling, still matching the exact mathematical operator as defined by RAFT. Our approach outperforms on-demand sampling by up to 92% while maintaining equally low memory usage, and performs at least on par with the default implementation with up to 99% lower memory usage. As cost sampling makes up a significant portion of the overall runtime, this can translate to up to 63% savings for the total end-to-end model inference on high-resolution inputs. Our evaluation of existing methods includes an 8K ultra-high-resolution dataset and an inference-time extension of the SEA-RAFT method. With this, we achieve state-of-the-art results at high resolutions both in accuracy and runtime.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。