提出新型混合代价体，大幅降低光流计算内存占用。
Hybrid Cost Volume for Memory-Efficient Optical Flow
- 用Top-k策略将4D代价体拆分为两个3D全局体，减少内存
- 引入局部4D代价体补充细节信息，保持高精度
- 在4K图像上仍低内存运行，适合实际部署场景
当前先进的光流方法大多基于密集的全对代价体，但随着图像分辨率提升，构建代价体的计算与空间复杂度呈四次方增长，难以应用于高分辨率图像。本文提出一种新型混合代价体（HCV），通过Top-k策略将4D代价体分解为两个全局3D代价体，显著降低内存占用并保留大量匹配信息；同时引入局部4D代价体，以小范围搜索空间补充局部细节。基于HCV设计了内存高效的光流网络HCVFlow。在Sintel、KITTI数据集及真实4K（2160×3840）图像上验证表明，HCVFlow相比基于全对代价体的递归方法，内存消耗大幅下降，且精度优于其他内存高效方法。代码已开源。
原文摘要 · Abstract (English)
Current state-of-the-art flow methods are mostly based on dense all-pairs cost volumes. However, as image resolution increases, the computational and spatial complexity of constructing these cost volumes grows at a quartic rate, making these methods impractical for high-resolution images. In this paper, we propose a novel Hybrid Cost Volume for memory-efficient optical flow, named HCV. To construct HCV, we first propose a Top-k strategy to separate the 4D cost volume into two global 3D cost volumes. These volumes significantly reduce memory usage while retaining a substantial amount of matching information. We further introduce a local 4D cost volume with a local search space to supplement the local information for HCV. Based on HCV, we design a memory-efficient optical flow network, named HCVFlow. Compared to the recurrent flow methods based the all-pairs cost volumes, our HCVFlow significantly reduces memory consumption while ensuring high accuracy. We validate the effectiveness and efficiency of our method on the Sintel and KITTI datasets and real-world 4K (2160*3840) resolution images. Extensive experiments show that our HCVFlow has very low memory usage and outperforms other memory-efficient methods in terms of accuracy. The code is publicly available at https://github.com/gangweiX/HCVFlow.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。