arXiv:2607.12297cs.CV2026-07中稿 · ECCV

将大模型SAM2蒸馏为轻量版,可在手机上实现图像视频分割。

MobileSAM2: Lightweight Segment Anything for Spatial Intelligence

论文配图:MobileSAM2: Lightweight Segment Anything for Spatial Intelligence
图 1 · 摘自论文原文
  • 用超图结构建模知识,分时序与多粒度两类蒸馏。
  • 在多个基准上表现良好,支持智能体任务的泛化能力。
  • 适合移动端部署,兼顾速度与分割精度。

近期的大规模视频基础模型SAM2可在图像和视频中实现任意分割,成为多种应用的强大基础模型。然而,许多实际场景需在手机、笔记本等资源受限设备上运行。本文旨在通过将重型SAM2蒸馏为轻量模型,使其实现在移动设备上的图像与视频任意分割。为此,提出超图知识蒸馏(HyperKD),首次将超图引入知识蒸馏,以有效建模并迁移SAM2的通用性与综合性知识。HyperKD包含时序超图蒸馏(Temporal HyperKD)与粒度超图蒸馏(Granularity HyperKD),分别显式建模并提取SAM2中的通用时序知识和全面的多粒度知识,并通过与构建的超图对齐,将其蒸馏至轻量学生模型。此外,提出MobileSAM2,一个通过超图蒸馏过程搜索最优架构的轻量级SAM2系列,在模型压缩中平衡效率与效果。大量实验验证了MobileSAM2在多个基准上的表现,且在具身智能任务中展现出优异的泛化性能。

原文摘要 · Abstract (English)

The recent large video foundation model, SAM2, enables segment anything in both images and videos, serving as a powerful base model for various applications. However, many of such use cases require to operate on resource-constrained devices like mobile phones and laptops. In this work, we aim to make SAM2 more mobile-friendly by distilling the heavyweight SAM2 into a lightweight model, facilitating segment anything in both images and videos on mobile devices. To this end, we propose Hypergraphical Knowledge Distill (HyperKD), which introduces the idea of hypergraph into knowledge distillation, aiming to effectively model and transfer SAM2's generalizable and comprehensive knowledge. HyperKD consists of Temporal HyperKD and Granularity HyperKD that construct hypergraphs to explicitly model and extract the generalizable temporal knowledge and the comprehensive multi-granularity knowledge from SAM2 respectively, which are then distilled into the lightweight student model by aligning it with the constructed hypergraphs. Besides, we present MobileSAM2, a new family of lightweight SAM2 that balances efficiency and effectiveness via searching the best model architectures with HyperKD during model size reduction. Extensive experiments validate MobileSAM2 across multiple benchmarks and show promising generalization performance on embodied AI tasks.

轻量化知识蒸馏图像分割移动端

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。