MOBIUS让大模型在手机上高效实现精准目标分割。
MOBIUS: Big-to-Mobile Universal Instance Segmentation via Multi-modal Bottleneck Fusion and Calibrated Decoder Pruning

- 用多模态瓶颈解码器融合多尺度信息,提升效率
- 训练迭代减少至1/3,计算量降低55%-75%仍保持顶尖性能
- 适合想在手机端部署高精度分割的开发者
扩大模型规模和训练数据已推动实例级感知的基础模型发展,在目标检测与分割任务中达到前沿的域内与零样本性能。然而其高计算开销限制了在资源受限平台的应用。本文首先分析现有架构在边缘部署中的局限性,提出MOBIUS系列基础模型,专为通用实例分割设计,可在从高端加速器到移动硬件的全设备范围内实现帕累托最优压缩。为降低训练与推理开销,提出:(i) 基于瓶颈的像素解码器,实现高效的多尺度、多模态融合;(ii) 基于语言引导的不确定性校准损失,实现自适应解码器剪枝;(iii) 简化统一的联合训练策略。相比高效基线模型以牺牲准确率为代价缩减复杂度,MOBIUS将像素解码器和Transformer解码器的FLOPs分别降低最多55%和75%,仅需三分之一训练迭代即可维持最先进性能。MOBIUS在高性能计算平台与移动设备上均建立了新的高效分割基准。
原文摘要 · Abstract (English)
Scaling up model size and training data has advanced foundation models for instance-level perception, achieving state-of-the-art in-domain and zero-shot performance across object detection and segmentation. However, their high computational cost limits adoption on resource-constrained platforms. We first examine the limitations of existing architectures in enabling efficient edge deployment without compromising performance. We then introduce MOBIUS, a family of foundation models for universal instance segmentation, designed for Pareto-optimal downscaling to support deployment across devices ranging from high-end accelerators to mobile hardware. To reduce training and inference demands, we propose: (i) a bottleneck pixel decoder for efficient multi-scale and multi-modal fusion, (ii) a language-guided uncertainty calibration loss for adaptive decoder pruning, and (iii) a streamlined, unified training strategy. Unlike efficient baselines that trade accuracy for reduced complexity, MOBIUS reduces pixel and transformer decoder FLOPs by up to 55% and 75%, respectively, while maintaining state-of-the-art performance in just a third of the training iterations. MOBIUS establishes a new benchmark for efficient segmentation on both high-performance computing platforms and mobile devices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。