用运动矢量传播检测结果，零样本视频目标检测更高效
MVP: Motion Vector Propagation for Zero-Shot Video Object Detection
- 仅在关键帧运行检测器，通过压缩域运动矢量传播结果
- 在ILSVRC2015-VID上达[email protected]=0.609，接近逐帧检测性能
- 无需标签、无需微调，适合零样本视频分析场景
在每帧运行大型开放词汇检测器虽准确但成本高。我们提出一种无需训练的流水线：仅在固定间隔的关键帧上运行OWLv2，利用压缩域运动矢量（MV）将检测结果传播至中间帧。通过3×3网格聚合运动矢量实现平移与均匀缩放更新，辅以面积增长检查和可选单类切换。该方法无需标签、无需微调，且对所有开放词汇方法使用同一提示列表。在ILSVRC2015-VID验证集上，所提方法（MVP）达到[email protected]=0.609，mAP@[0.5:0.95]=0.316。在宽松交并比阈值下，性能接近逐帧OWLv2-Large（0.747/0.721 vs 0.784/0.780，IoU=0.2/0.3），表明粗略定位基本保留。相同关键帧调度下，MVP优于基于追踪器的传播方法（MOSSE、KCF、CSRT）。监督基线YOLOv12x在[email protected]上达0.631，但需标注训练；而本方法保持无标签与开放词汇特性。结果表明，压缩域传播是降低检测调用次数同时保持强零样本覆盖的实用方案。
原文摘要 · Abstract (English)
Running a large open-vocabulary (Open-vocab) detector on every video frame is accurate but expensive. We introduce a training-free pipeline that invokes OWLv2 only on fixed-interval keyframes and propagates detections to intermediate frames using compressed-domain motion vectors (MV). A simple 3x3 grid aggregation of motion vectors provides translation and uniform-scale updates, augmented with an area-growth check and an optional single-class switch. The method requires no labels, no fine-tuning, and uses the same prompt list for all open-vocabulary methods. On ILSVRC2015-VID (validation dataset), our approach (MVP) attains [email protected]=0.609 and mAP@[0.5:0.95]=0.316. At loose intersection-over-union (IoU) thresholds it remains close to framewise OWLv2-Large (0.747/0.721 at 0.2/0.3 versus 0.784/0.780), reflecting that coarse localization is largely preserved. Under the same keyframe schedule, MVP outperforms tracker-based propagation (MOSSE, KCF, CSRT) at [email protected]. A supervised reference (YOLOv12x) reaches 0.631 at [email protected] but requires labeled training, whereas our method remains label-free and open-vocabulary. These results indicate that compressed-domain propagation is a practical way to reduce detector invocations while keeping strong zero-shot coverage in videos. Our code and models are available at https://github.com/microa/MVP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。