用焦点化分块策略，让点提示分割更高效
Segment This Thing: Foveated Tokenization for Efficient Point-Prompted Segmentation
- 以提示点为中心，越远区域越低分辨率分块
- 令牌数比均匀分块少得多，计算量大幅降低
- 适合需要实时响应的增强现实与机器人场景
本文提出Segment This Thing（STT），一种基于单点提示生成单一分割结果的高效图像分割模型。不同于以往通过缩小模型规模提升效率的做法，STT通过聚焦输入图像实现高效计算：给定图像和一个点提示后，模型以该点为中心提取裁剪区域，并采用新型变分辨率分块策略，分块下采样率随距离提示点的距离增加而提高。该方法产生的图像令牌数量远低于均匀分块，从而显著降低计算成本，且无需缩减模型规模。同时，焦点化机制使模型集中关注感兴趣区域，带来有益的归纳偏置。实验表明，STT在分割基准上保持竞争力，同时运行速度远超已有方法，可在消费级硬件上实现交互式帧率，适用于增强现实或机器人等实时应用。
原文摘要 · Abstract (English)
This paper presents Segment This Thing (STT), a new efficient image segmentation model designed to produce a single segment given a single point prompt. Instead of following prior work and increasing efficiency by decreasing model size, we gain efficiency by foveating input images. Given an image and a point prompt, we extract a crop centered on the prompt and apply a novel variable-resolution patch tokenization in which patches are downsampled at a rate that increases with increased distance from the prompt. This approach yields far fewer image tokens than uniform patch tokenization. As a result we can drastically reduce the computational cost of segmentation without reducing model size. Furthermore, the foveation focuses the model on the region of interest, a potentially useful inductive bias. We show that our Segment This Thing model is more efficient than prior work while remaining competitive on segmentation benchmarks. It can easily run at interactive frame rates on consumer hardware and is thus a promising tool for augmented reality or robotics applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。