用隐式神经特征匹配,高效跨视频传播医学影像标注。
Match4Annotate: Propagating Sparse Video Annotations via Implicit Neural Feature Matching
- 测试时用SIREN拟合DINOv3特征,生成连续时空特征场。
- 在三个临床超声数据集上超越现有跨视频标注方法。
- 轻量级设计适合医疗等专业领域快速标注需求。
获取每帧视频标注仍是医学影像等专业领域部署计算机视觉的主要瓶颈,专家标注耗时且成本高。标签传播提供自然解决方案,但现有方法存在根本局限:视频追踪器和分割模型仅能单视频内传播,需逐视频初始化且无法跨视频泛化;经典对应管线依赖检测器选取关键点,在低纹理场景表现差;而密集特征匹配与单次分割方法虽支持跨视频传播,却缺乏时空平滑性,且不统一支持点与掩码标注。我们提出Match4Annotate,一种轻量级框架,实现视频内与跨视频的点与掩码标注传播。方法在测试时用SIREN拟合DINOv3特征,生成连续、高分辨率的时空特征场,并学习帧间平滑隐式形变场以引导对应匹配。在三个挑战性临床超声数据集上评估,Match4Annotate在跨视频传播中达到最先进性能,优于特征匹配与单次分割基线,同时在视频内传播中保持与专用追踪器相当的竞争力。结果表明,轻量级、测试时优化的特征匹配流水线有望为可扩展标注流程提供高效且易用的解决方案。
原文摘要 · Abstract (English)
Acquiring per-frame video annotations remains a primary bottleneck for deploying computer vision in specialized domains such as medical imaging, where expert labeling is slow and costly. Label propagation offers a natural solution, yet existing approaches face fundamental limitations. Video trackers and segmentation models can propagate labels within a single sequence but require per-video initialization and cannot generalize across videos. Classic correspondence pipelines operate on detector-chosen keypoints and struggle in low-texture scenes, while dense feature matching and one-shot segmentation methods enable cross-video propagation but lack spatiotemporal smoothness and unified support for both point and mask annotations. We present Match4Annotate, a lightweight framework for both intra-video and inter-video propagation of point and mask annotations. Our method fits a SIREN-based implicit neural representation to DINOv3 features at test time, producing a continuous, high-resolution spatiotemporal feature field, and learns a smooth implicit deformation field between frame pairs to guide correspondence matching. We evaluate on three challenging clinical ultrasound datasets. Match4Annotate achieves state-of-the-art inter-video propagation, outperforming feature matching and one-shot segmentation baselines, while remaining competitive with specialized trackers for intra-video propagation. Our results show that lightweight, test-time-optimized feature matching pipelines have the potential to offer an efficient and accessible solution for scalable annotation workflows.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。