arXiv:2502.02763cs.CV2025-02被引 2

用局部聚焦的视觉变压器实现高效精准分割,参数少440倍仍更优。

Looking Locally: Object-Centric Vision Transformers as Foundation Models for Efficient Segmentation

  • 基于生物启发的自上而下注意力,只对目标物体周围区域高分辨率处理。
  • 小模型(0.51M参数)达79.9%平均交并比,优于大模型SAM2(224M参数)。
  • 擅长小物体分割,适合实时场景跟踪与低功耗应用。

当前先进分割模型先编码整图再聚焦对象,浪费算力。本文提出FLIP(视网膜样输入分块),一种参数高效的视觉模型,通过生物启发的自上而下注意力实现对象分割。FLIP从输入图像中选择性采样围绕目标对象的多分辨率块,使对象中心获得高分辨率处理,同时保持粗粒度周边上下文。这种非网格、尺度不变的设计让其在六项基准测试中显著领先:在五个主流数据集(Hypersim, KITTI-360, OpenImages, COCO, LVIS)上持续超越SAM及其变体;在新提出的ObjaScale数据集(对象面积占比0.0001%至25%)中,对极小物体仍能准确分割,而现有模型严重失效。FLIP-Tiny(0.51M参数)达79.90%平均交并比,远超SAM2-L(224.45M参数,75.87%);FLIP-Large(96.6M参数)达83.26%,且运行速度快约2倍。该模型具高能效,适合作为时序追踪等任务的基座模型。

原文摘要 · Abstract (English)

Current state-of-the-art segmentation models encode entire images before focusing on specific objects. This wastes computational resources. We introduce FLIP (Fovea-Like Input Patching), a parameter-efficient vision model that realizes object segmentation through biologically-inspired top-down attention. FLIP selectively samples multi-resolution patches centered on objects of interest from the input. As a result, it allocates high-resolution processing to object centers while maintaining coarser peripheral context. This off-grid, scale-invariant design enables FLIP to outperform META's Segment Anything models (SAM, SAM2 and fast variants) by large margins: With more than 440$\times$ fewer parameters, FLIP-Tiny (0.51M parameters) reaches a mean IoU of 79.90% while SAM2-L reaches 75.87% IoU (224.45M parameters). FLIP-Large even achieves 83.26% mean IoU (96.6M parameters), still running about $2\times$ faster than SAM2-L. We evaluate on six benchmarks in total. In five established benchmarks (Hypersim, KITTI-360, OpenImages, COCO, LVIS) FLIP consistently outperforms SAM and various variants of it. In our novel ObjaScale dataset, which stress-tests scale invariance with objects ranging from 0.0001% up to 25% of the image area, we show that FLIP segments even very small objects accurately, where existing models fail severely. FLIP opens new possibilities for real-time, object-centric vision and offers much higher energy efficiency. We believe that FLIP can act as a powerful foundation model, as it is very well-suited to track objects over time, for example, when being integrated into slot-based scene segmentation architectures.

视觉分割注意力机制高效模型小物体检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。