用自然语言指令统一多种图像分割任务，生成更精准的分割掩码。
LlamaSeg: Image Segmentation via Autoregressive Mask Generation
- 将分割任务转为视觉生成，通过自回归方式预测掩码像素。
- 在200万标注掩码上训练，支持5800+开放词汇标签。
- 提出新评估指标，更适合衡量生成掩码的轮廓精度。
我们提出LlamaSeg，一种基于自然语言指令的统一图像分割框架。通过将分割重构为视觉生成任务，模型将掩码编码为视觉标记，并使用类LLaMA的Transformer进行直接的下一个标记预测，自然适配自回归架构。为支持大规模训练，我们设计数据标注流程，构建了包含200万分割掩码的SA-OVRS数据集，涵盖超过5800种开放词汇标签或多样化文本描述，覆盖真实世界多种场景。该模型可依据文本提示定位图像中的对象并生成细粒度掩码。我们进一步引入复合度量平均豪斯多夫距离（d_AHD），以更优评估生成模型的掩码轮廓保真度。实验表明，LlamaSeg在多个分割基准上持续优于现有生成方法，输出更精细、更准确的分割结果。代码与数据集见：https://github.com/GML-FMGroup/llamaseg。
原文摘要 · Abstract (English)
We present \textbf{LlamaSeg}, a visual autoregressive framework that unifies multiple image segmentation tasks via natural language instructions. By reformulating segmentation as visual generation, LlamaSeg encodes masks as visual tokens and uses a LLaMA-style Transformer for direct next-token prediction, naturally fitting segmentation into autoregressive architectures. To support large-scale training, we introduce a data annotation pipeline and construct the \textbf{SA-OVRS} dataset, which contains \textbf{2M} segmentation masks annotated with over \textbf{5,800} open vocabulary labels or diverse textual descriptions, spanning diverse real-world scenarios. This enables our model to localize objects in images based on text prompts and to generate fine-grained masks. We further introduce the composite metric average Hausdorff Distance ($d_{\mathrm{AHD}}$) to evaluate mask contour fidelity for generative models better. Experiments show that LlamaSeg consistently outperforms existing generative approaches on multiple segmentation benchmarks and delivers finer, more accurate segmentation masks. Code and dataset are available at \href{https://github.com/GML-FMGroup/llamaseg}{https://github.com/GML-FMGroup/llamaseg}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。