用约束提示让SAM2实时分割光场图像，效果更好速度更快。
Segment Anything in Light Fields for Real-Time Applications via Constrained Prompting
- 利用光场几何约束传播掩码，不重训练模型
- 生成高质量且视图一致的光场分割掩码，速度快7倍
- 适合需要实时光场分割的应用，如物体姿态跟踪
分割光场图像可在计算机视觉任务中有效利用物体的几何与外观信息，如物体姿态跟踪。在光场领域，分割还需保证同一物体在所有视角下识别一致。虽然分割一切模型2（SAM 2）可为单目图像和视频生成语义合理分割，但直接应用于光场时效率极低，因未利用光场约束。本文提出一种新方法，无需重训练或修改模型，适配SAM 2至光场域。通过利用视点间极线几何线索传播掩码，探测SAM 2潜在空间估计遮挡，并进一步引导其优化，实现高保真、视图一致的光场掩码。相比SAM 2视频跟踪基线，该方法性能更优，且运行速度提升7倍，达到实时处理水平。
原文摘要 · Abstract (English)
Segmented light field images can serve as a powerful representation in many of computer vision tasks exploiting geometry and appearance of objects, such as object pose tracking. In the light field domain, segmentation presents an additional objective of recognizing the same segment through all the views. Segment Anything Model 2 (SAM 2) allows producing semantically meaningful segments for monocular images and videos. However, using SAM 2 directly on light fields is highly ineffective due to unexploited constraints. In this work, we present a novel light field segmentation method that adapts SAM 2 to the light field domain without retraining or modifying the model. By utilizing the light field domain constraints, the method produces high quality and view-consistent light field masks, outperforming the SAM 2 video tracking baseline and working 7 times faster, with a real-time speed. We achieve this by exploiting the epipolar geometry cues to propagate the masks between the views, probing the SAM 2 latent space to estimate their occlusion, and further prompting SAM 2 for their refinement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。