用帧差替代光流，实现低资源下的视频目标分割
Investigation of Frame Differences as Motion Cues for Video Object Segmentation
- 以帧差代替光流，降低计算开销
- 在固定摄像头视频上性能接近光流模型
- 适合边缘设备实时应用
自动视频目标分割（AVOS）是指在不依赖首帧人工标注的情况下，自动分割视频序列中的目标物体。在AVOS中，运动信息至关重要，通常使用光流来捕捉运动线索。然而，光流计算资源消耗大，不适合在计算资源有限的边缘设备上实现实时应用。本文提出用帧差作为光流的替代方案来提取运动线索。我们设计了一种类似U-Net的扩展模型，输入包括待分割帧和帧差，输出为估计的分割图。实验结果表明，该模型在固定摄像头拍摄的视频上性能与使用光流的模型相当，验证了帧差在计算资源受限场景下作为运动线索的有效性。
原文摘要 · Abstract (English)
Automatic Video Object Segmentation (AVOS) refers to the task of autonomously segmenting target objects in video sequences without relying on human-provided annotations in the first frames. In AVOS, the use of motion information is crucial, with optical flow being a commonly employed method for capturing motion cues. However, the computation of optical flow is resource-intensive, making it unsuitable for real-time applications, especially on edge devices with limited computational resources. In this study, we propose using frame differences as an alternative to optical flow for motion cue extraction. We developed an extended U-Net-like AVOS model that takes a frame on which segmentation is performed and a frame difference as inputs, and outputs an estimated segmentation map. Our experimental results demonstrate that the proposed model achieves performance comparable to the model with optical flow as an input, particularly when applied to videos captured by stationary cameras. Our results suggest the usefulness of employing frame differences as motion cues in cases with limited computational resources.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。