用预训练模型直接算光流,不迭代也能跑得准。
Rethinking Dense Optical Flow without Test-Time Scaling

- 用冻结的DINO-v2和单目深度模型提取语义与几何先验
- 单次前向传播在Sintel上达2.81 EPE,超越多个强基线
- 适合追求高效推理、不想加测试时计算的场景
近年来密集光流方法依赖复杂的架构和多步迭代优化来提升性能,但需大量推理计算。本文提出新框架,无需测试时迭代或额外计算,仅通过一次前向传播即可估计密集光流。该方法从冻结的DINO-v2骨干网络提取视觉语义特征,并融合来自单目深度基础模型的几何线索,构建统一表示,采用全局匹配方式直接求解稠密对应关系。尽管不进行迭代优化,其在多个挑战性基准上仍表现出色:在Sintel Final数据集上达到2.81 EPE,优于相同训练条件下SOTA方法SEA-RAFT,且显著领先于无迭代的RAFT与GMFlow。结果表明,强大的基础模型先验可替代测试时扩展,为高效光流提供新路径。
原文摘要 · Abstract (English)
Recent progress in dense optical flow has been driven by increasingly complex architectures and multi-step refinement for test-time scaling. While these approaches achieve strong benchmark performance, they also require substantial computation during inference. This raises a fundamental question: Is scaling test-time computation the only way to improve dense optical flow accuracy? We argue that it is not. Instead, powerful visual semantic and geometric priors encoded in modern foundation models can reduce, if not overcome, the need for computationally expensive iterative refinement at test-time. In this paper, we present a framework that estimates dense optical flow in a single forward pass, leveraging pretrained foundation representations, while avoiding iterative refinement and additional inference-time computation, thus offering an alternative to test-time scaling. Our method extracts visual semantic features from a frozen DINO-v2 backbone and combines them with geometric cues from a monocular depth foundation model. We fuse these complementary priors into a unified representation and apply a global matching formulation to estimate dense correspondences without recurrent updates or test-time optimization. Despite avoiding iterative refinement, our approach achieves strong cross-dataset generalization across challenging benchmarks. On Sintel Final, we obtain 2.81 EPE without refinement, significantly improving over state-of-the-art (SOTA) SEA-RAFT under comparable training conditions and outperforming RAFT, GMFlow (without refinement), and recent FlowSeek in the same setting. These results suggest that strong foundation priors can substitute for test-time scaling, offering a computationally efficient alternative to refinement-heavy pipelines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。