用轨迹和语义融合实现视频中任意运动物体精准分割。
Segment Any Motion in Videos
- 结合长时轨迹与语义特征，提升运动感知能力。
- 在多个数据集上达到当前最佳效果，尤其擅长细粒度分割。
- 适合需要高精度视频分割的科研与工业应用。
运动物体分割是实现视觉场景高层理解的关键任务，具有广泛下游应用。人类可轻松识别视频中的运动物体。以往方法主要依赖光流提供运动线索，但常因局部运动、复杂形变、运动模糊和背景干扰导致预测不准确。本文提出一种新方法，融合长时轨迹运动线索与基于DINO的语义特征，并利用SAM2通过迭代提示策略实现像素级掩码细化。模型采用时空轨迹注意力与运动-语义解耦嵌入，优先关注运动信息同时整合语义支持。在多样数据集上的大量测试表明，该方法在挑战性场景和多物体细粒度分割上均表现卓越，达到当前最优水平。代码已公开：https://motion-seg.github.io/。
原文摘要 · Abstract (English)
Moving object segmentation is a crucial task for achieving a high-level understanding of visual scenes and has numerous downstream applications. Humans can effortlessly segment moving objects in videos. Previous work has largely relied on optical flow to provide motion cues; however, this approach often results in imperfect predictions due to challenges such as partial motion, complex deformations, motion blur and background distractions. We propose a novel approach for moving object segmentation that combines long-range trajectory motion cues with DINO-based semantic features and leverages SAM2 for pixel-level mask densification through an iterative prompting strategy. Our model employs Spatio-Temporal Trajectory Attention and Motion-Semantic Decoupled Embedding to prioritize motion while integrating semantic support. Extensive testing on diverse datasets demonstrates state-of-the-art performance, excelling in challenging scenarios and fine-grained segmentation of multiple objects. Our code is available at https://motion-seg.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。