arXiv:2504.06863cs.CV2025-04被引 1

用深度思考让单张图识别运动物体,突破无时间信息限制。

MovSAM: A Single-image Moving Object Segmentation Framework Based on Deep Thinking

  • 通过思维链提示激活多模态大模型,生成逻辑推理驱动的分割提示。
  • 在公开数据集上达到92.5%的J&F指标,超越传统多帧方法。
  • 适用于自动驾驶等无法依赖连续帧的场景,具实用价值。

运动物体分割在理解动态视觉环境方面至关重要。现有方法依赖多帧图像序列识别运动物体,而单张图分割对运动意图预测和应对相机丢帧等应用尤为关键。然而,由于缺乏时序线索,单张图运动物体分割仍面临挑战。为此,我们提出MovSAM,首个面向单张图像的运动物体分割框架。MovSAM利用增强思维链(CoT)提示的多模态大语言模型(MLLM),通过深度思考搜索运动物体并生成文本提示,与分割任意模型(SAM)和视觉语言模型(VLM)的视觉特征进行跨模态融合,实现基于逻辑推理的分割。分割结果经过深度思考优化循环,使MovSAM能迭代深化对场景上下文及物体间关系的理解。该方法在真实世界自动驾驶场景中验证了实用性。尽管多帧方法具备时间信息优势,MovSAM仍达到主流基准上92.5%的J&F性能,处于领先水平。代码将开源于https://github.com/IRMVLab/MovSAM。

原文摘要 · Abstract (English)

Moving object segmentation plays a vital role in understanding dynamic visual environments. While existing methods rely on multi-frame image sequences to identify moving objects, single-image MOS is critical for applications like motion intention prediction and handling camera frame drops. However, segmenting moving objects from a single image remains challenging for existing methods due to the absence of temporal cues. To address this gap, we propose MovSAM, the first framework for single-image moving object segmentation. MovSAM leverages a Multimodal Large Language Model (MLLM) enhanced with Chain-of-Thought (CoT) prompting to search the moving object and generate text prompts based on deep thinking for segmentation. These prompts are cross-fused with visual features from the Segment Anything Model (SAM) and a Vision-Language Model (VLM), enabling logic-driven moving object segmentation. The segmentation results then undergo a deep thinking refinement loop, allowing MovSAM to iteratively improve its understanding of the scene context and inter-object relationships with logical reasoning. This innovative approach enables MovSAM to segment moving objects in single images by considering scene understanding. We implement MovSAM in the real world to validate its practical application and effectiveness for autonomous driving scenarios where the multi-frame methods fail. Furthermore, despite the inherent advantage of multi-frame methods in utilizing temporal information, MovSAM achieves state-of-the-art performance across public MOS benchmarks, reaching 92.5\% on J\&F. Our implementation will be available at https://github.com/IRMVLab/MovSAM.

单图分割运动目标大模型自动驾驶

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。