arXiv:2607.27902cs.CV2026-07中稿 · ACM Multimedia 202…

用单个视觉标记精准定位场景文字,提升小文本和密集文本识别效果。

One Patch Is Enough: Reinforcement-Optimized Visual Token Grounding for MLLM-Based Scene Text Spotting

论文配图:One Patch Is Enough: Reinforcement-Optimized Visual Token Grounding for MLLM-Based Scene Text Spotting
图 1 · 摘自论文原文
  • 每行文字只选一个关键视觉标记,减少噪声干扰。
  • 在ICDAR2015上达到86.3%的F-measure,超越主流模型。
  • 适合需要高精度文字定位的工业级应用。

场景文字检测要求文本识别与空间定位高度精确对齐。尽管视觉标记定位已成为多模态大模型(MLLM)的有前景范式,但以往的多块区域方法常引入冗余噪声和定位模糊,尤其在密集或小尺寸文本上表现不佳。为此,我们提出单块文字检测(SPaTS),一种以视觉为中心的框架:将每个文本实例通过单一锚点视觉标记进行路由,并通过全图细化恢复几何信息。为在无标签情况下准确识别该锚点,我们引入单块选择优化(SPaSO),一种基于补丁级别奖励的强化学习框架,用于优化离散视觉标记选择。为进一步提升表示鲁棒性与定位精度,我们提出方向嵌入对齐(DEA),通过解耦特征幅度与方向,抑制不稳定的范数偏差;并设计补丁增强解码(PED),将路由锚点与语言语义融合,跨注意力全图特征图,实现超越坐标空间代理的几何感知边界回归。大量实验表明,SPaTS在多个基准上持续显著优于前沿闭源MLLM和专用OCR MLLM。代码已开源:https://github.com/eeNickTang/SPaTS。

原文摘要 · Abstract (English)

Scene text spotting requires high-precision alignment between textual recognition and spatial localization. While visual-token grounding has emerged as a promising formulation for Multimodal Large Language Models (MLLMs), the previous multi-patch paradigm often introduces redundant noise and localization ambiguity, particularly for dense or small text instances. To address this, we propose Single-Patch Text Spotting (SPaTS), a vision-centric framework that routes each text instance through a single anchor visual token and then recovers geometry via full-image refinement. To accurately identify this anchor without oracle labels, we introduce Single-Patch Selective Optimization (SPaSO), a reinforcement learning framework that optimizes discrete visual-token selection using patch-level rewards. To further improve representation robustness and localization precision, we introduce Directional Embedding Alignment (DEA) to suppress unstable norm bias by decoupling feature magnitude and direction, and Patch-Enhanced Decoding (PED) to fuse the routed anchor with language semantics and cross-attend over the full-image feature map for geometry-aware boundary regression beyond coordinate-space surrogates. Extensive experiments demonstrate that SPaTS consistently and significantly outperforms both frontier closed-source MLLMs and OCR MLLMs. Code is available at https://github.com/eeNickTang/SPaTS.

文字检测多模态强化学习视觉定位

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。