AV1运动矢量可高效替代传统光流，加速视觉任务计算。
AV1 Motion Vector Fidelity and Application for Efficient Optical Flow
- 从AV1编码视频提取运动矢量，作为光流的初始估计。
- 使用AV1矢量作为起点，RAFt模型收敛速度提升4倍，误差仅轻微增加。
- 适合需要快速运动估计的实时视觉应用，如视频分析与压缩优化。
本文系统分析了从AV1编码视频流中提取的运动矢量及其在加速光流估计中的应用。研究表明，AV1运动矢量可作为高质量且计算高效的传统光流替代方案，后者是许多计算机视觉流水线中关键但资源消耗大的组件。主要贡献有两点：首先，对比了AV1与HEVC的运动矢量与真实光流的保真度，揭示了编码器设置对运动估计精度的影响，并提出最优配置建议；其次，将提取的AV1运动矢量用作先进深度学习光流方法RAFT的“热启动”，显著降低收敛时间，实现四倍计算速度提升，端点误差仅轻微增加。结果表明，重用压缩视频中的运动矢量是一种广泛适用于各类运动感知视觉任务的实用高效方法。
原文摘要 · Abstract (English)
This paper presents a comprehensive analysis of motion vectors extracted from AV1-encoded video streams and their application in accelerating optical flow estimation. We demonstrate that motion vectors from AV1 video codec can serve as a high-quality and computationally efficient substitute for traditional optical flow, a critical but often resource-intensive component in many computer vision pipelines. Our primary contributions are twofold. First, we provide a detailed comparison of motion vectors from both AV1 and HEVC against ground-truth optical flow, establishing their fidelity. In particular we show the impact of encoder settings on motion estimation fidelity and make recommendations about the optimal settings. Second, we show that using these extracted AV1 motion vectors as a "warm-start" for a state-of-the-art deep learning-based optical flow method, RAFT, significantly reduces the time to convergence while achieving comparable accuracy. Specifically, we observe a four-fold speedup in computation time with only a minor trade- off in end-point error. These findings underscore the potential of reusing motion vectors from compressed video as a practical and efficient method for a wide range of motion-aware computer vision applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。