轻量级模型实现传感器端实时目标分割，精度高且低延迟。
PicoSAM3: Real-Time In-Sensor Region-of-Interest Segmentation
- 结合密集卷积与提示编码，支持灵活区域分割
- 在COCO和LVIS上分别达65.45%和64.01% mIoU
- 可在索尼IMX500传感器上以11.82ms延迟实时运行
实时、本地化分割对智能眼镜和物联网设备等低延迟、隐私敏感场景至关重要。我们提出PicoSAM3，一种专为边缘和传感器端部署优化的轻量级可提示视觉分割模型，支持在索尼IMX500视觉传感器上运行。PicoSAM3仅含130万参数，融合密集卷积架构、感兴趣区域提示编码、高效通道注意力机制，并通过SAM2和SAM3的知识蒸馏提升性能。在COCO和LVIS数据集上，mIoU分别达到65.45%和64.01%，优于同类基于SAM的轻量级及边缘专用模型，且复杂度相当或更低。量化至INT8后精度几乎无损，可在IMX500上实现11.82毫秒延迟的实时推理，完全符合其内存与算子约束。消融实验表明，从大型SAM模型蒸馏可带来最高+14.5% mIoU提升，验证了在传感器层面实现高质量、空间灵活的可提示分割的可行性。
原文摘要 · Abstract (English)
Real-time, on-device segmentation is critical for latency-sensitive and privacy-aware applications such as smart glasses and Internet-of-Things devices. We introduce PicoSAM3, a lightweight promptable visual segmentation model optimized for edge and in-sensor execution, including deployment on the Sony IMX500 vision sensor. PicoSAM3 has 1.3M parameters and combines a dense CNN architecture with region of interest prompt encoding, Efficient Channel Attention, and knowledge distillation from SAM2 and SAM3. On COCO and LVIS, PicoSAM3 achieves 65.45% and 64.01% mIoU, respectively, outperforming existing SAM-based and edge-oriented baselines at similar or lower complexity. The INT8 quantized model preserves accuracy with negligible degradation while enabling real-time in-sensor inference at 11.82ms latency on the IMX500, fully complying with its memory and operator constraints. Ablation studies show that distillation from large SAM models yields up to +14.5% mIoU improvement over supervised training and demonstrate that high-quality, spatially flexible promptable segmentation is feasible directly at the sensor level.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。