无需预训练标签，仅用一帧掩码即可追踪被遮挡物体。
Track Anything Behind Everything: Zero-Shot Amodal Video Object Segmentation
- 用可见部分掩码作为查询，通过扩散模型生成不可见部分。
- 测试时微调模型，实现零样本追踪与完整补全。
- 适合需要灵活追踪任意物体的场景，如自动驾驶、机器人视觉。
我们提出 Track Anything Behind Everything（TABE），一种全新的零样本非可视视频目标分割方法。不同于依赖预训练类别标签的现有方法，本方法仅需第一帧中物体可见区域的一个查询掩码，即可实现灵活的零样本推理。我们将非可视分割建模为从可见掩码出发的生成式外推任务，利用预训练视频扩散模型完成。无需重新训练扩散模型以增加输入通道，而是直接在测试时微调预训练模型，使其专用于追踪特定对象。该方法特别设计用于处理完全遮挡情况下的非可视补全。模型与代码将全部公开。
原文摘要 · Abstract (English)
We present Track Anything Behind Everything (TABE), a novel pipeline for zero-shot amodal video object segmentation. Unlike existing methods that require pretrained class labels, our approach uses a single query mask from the first frame where the object is visible, enabling flexible, zero-shot inference. We pose amodal segmentation as generative outpainting from modal (visible) masks using a pretrained video diffusion model. We do not need to re-train the diffusion model to accommodate additional input channels but instead use a pretrained model that we fine-tune at test-time to allow specialisation towards the tracked object. Our TABE pipeline is specifically designed to handle amodal completion, even in scenarios where objects are completely occluded. Our model and code will all be released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。