arXiv:2608.13980cs.CV2026-08

提出细粒度令牌内掩码表示,提升遥感图像分割精度

FIRM: Fine-Grained Intra-Token Representation of Masks for Remote Sensing Reasoning Segmentation

论文配图:FIRM: Fine-Grained Intra-Token Representation of Masks for Remote Sensing Reasoning Segmentation
图 1 · 摘自论文原文
  • 为每个视觉令牌预测子网格掩码模式,而非单一标签
  • 在LaSeRS上达70.5/80.5 gIoU/cIoU,EarthReason平均提升3.0点
  • 适合需要高精度分割的遥感目标识别任务

推理分割要求多模态大语言模型将隐含指令转化为精确的像素级掩码。现有方法将图像编码为视觉令牌,每个令牌合并一组图像块。在遥感图像中,小目标、细结构和相邻实例可能分布于同一令牌的不同区域。对这类令牌赋予单一二值标签会丢失内部空间结构,导致邻近目标合并、边界粗糙。为此,本文提出FIRM(细粒度令牌内掩码表示),对每个视觉令牌预测一个r×r的二值子单元模式,而非单一前景/背景标签。在识别目标后,通过一次掩码前向过程预测完整的掩码代码网格。固定查找将代码转换为离散子单元掩码,而对代码分布进行边缘化可得软结构场。为进一步恢复子单元内的精细边界,引入轻量级连续渲染器,利用预合并视觉特征与图像细节优化该场。在五个卫星与无人机图像的推理与指代分割基准上,FIRM取得领先结果,包括在LaSeRS上达到70.5/80.5 gIoU/cIoU,EarthReason平均提升3.0点。结果表明显式表征令牌内掩码模式对细粒度多模态大模型分割具有重要价值。

原文摘要 · Abstract (English)

Reasoning segmentation requires multimodal large language models (MLLMs) to translate implicit instructions into precise pixel-level masks. MLLMs encode an image as visual tokens, each of which merges a group of image patches. In remote sensing images, small targets, thin structures, and adjacent instances can occupy different parts of the same visual token. Assigning a single binary mask label to such a token loses its internal spatial structure, causing nearby targets to merge and object boundaries to become coarse. To bridge this representational gap, we introduce FIRM, a Fine-grained Intra-token Representation of Masks. For each visual token, FIRM predicts a mask code that specifies an $r\times r$ binary sub-cell pattern rather than a single foreground/background label. Given a target identified by the MLLM, the complete grid of mask codes is predicted in one mask pass. Fixed lookup converts the predicted codes into a discrete sub-cell mask, while marginalizing the code distribution yields a soft structural field. To further recover fine-grained boundaries within each sub-cell, we introduce a lightweight continuous renderer that refines this field using pre-merge visual features and image details. Across five reasoning and referring segmentation benchmarks on satellite and UAV images, FIRM achieves leading results, including $70.5/80.5$ gIoU/cIoU on LaSeRS and a $3.0$-point average gain on EarthReason. These results demonstrate the value of explicitly representing intra-token mask patterns for fine-grained MLLM segmentation.

遥感分割多模态模型细粒度分割

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。