arXiv:2605.23428cs.CVcs.MM2026-05

用语义感知提前停机,让视频运动估计更省算力

FAST-ME: Foundation-aware Adaptive Stopping for Motion Estimation for Efficient IoT Video Analysis

论文配图:FAST-ME: Foundation-aware Adaptive Stopping for Motion Estimation for Efficient IoT Video Analysis
图 1 · 摘自论文原文
  • 结合视觉基础模型与最优停止理论,动态判断何时终止运动估计
  • 计算量降低60%以上,精度损失低于1%,且关键区域覆盖更好
  • 适合资源受限的物联网视频分析,尤其是智能监控与边缘计算

在物联网摄像头网络等资源受限场景中,高效视频处理至关重要。块运动估计(ME)虽有快速搜索技术,仍存在计算开销大问题。本文提出基于最优停止理论(OST)的运动估计算法,通过分析帧内与帧间时空差异进行评估。同时设计语义感知框架,融合视觉基础模型(如ViT、SAM)生成的语义注意力分数,与传统误差度量(如SAD)结合,形成混合停止准则。该方法在冗余区域提前终止搜索,在语义重要区域持续计算。在基准与多模态视频数据集上的实验表明,该方法在计算量降低超过60%的情况下,精度损失小于1%,且显著提升关键区域的语义覆盖能力。结果证明,低层运动分析与高层语义推理的融合,为下一代智能系统的高效多模态视频理解提供了可行路径。

原文摘要 · Abstract (English)

In modern multimedia systems, efficient video processing is critical, especially in resource-constrained environments such as IoT-based camera networks, autonomous platforms, and wireless sensor multimedia systems. A key bottleneck in video compression and understanding is block motion estimation (ME), a process that remains computationally expensive despite the development of fast search techniques. This work introduces an Optimal Stopping Theory (OST) algorithm for block motion estimation based on the assessment of spatiotemporal differences within and across video frames. It also proposes a semantic-aware motion estimation framework that integrates Foundation Models (FMs) with the OST-based decision process. By leveraging pretrained visual models such as Vision Transformers (ViT) and the Segment Anything Model (SAM), the framework extracts semantic attention scores that indicate the importance of motion within specific spatial regions. These scores are fused with traditional distortion-based metrics, such as the Sum of Absolute Differences (SAD), to guide a hybrid stopping criterion that jointly considers motion magnitude and semantic relevance. The resulting adaptive algorithm stops early in redundant regions while continuing the search in areas where motion is semantically significant. Experiments compare the proposed solution with widely used approaches from the literature on benchmark and multimodal video datasets. The proposed method achieves a significant reduction in computation with minimal accuracy loss and improved semantic coverage. The results highlight the benefits of bridging low-level motion analysis with high-level semantic reasoning, offering a promising direction for efficient multimodal video understanding in next-generation smart systems.

运动估计边缘计算基础模型视频压缩

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。