arXiv:2510.18840cs.CVcs.CL2025-10被引 10

让AI像人一样看文字,用图像理解替代传统分词。

See the Text: From Tokenization to Visual Reading

  • 将文本转为图像,用多模态模型直接视觉阅读
  • 减少4.43倍令牌数,降低70.5%计算量
  • 提升跨语言泛化与抗错字、乱字体能力

人类阅读文字时先识别其形状、布局和模式,再理解意义,因此能有效应对拼写错误、变形字体和多种文字系统。而现代大语言模型依赖固定词汇表的子词分词,对低资源语言过度切分,产生冗长且无语言意义的序列,增加计算负担。本文提出新方法SeeTok,将文本渲染为视觉图像(visual-text),借助预训练多模态大模型进行解读,复用其在大规模图文对中学习到的强大光学字符识别(OCR)和文本-视觉对齐能力。在三种不同语言任务上,SeeTok性能达到或超过子词分词,同时令牌数减少4.43倍,浮点运算量(FLOPs)降低70.5%,并提升跨语言泛化能力、对排版噪声的鲁棒性及语言层级建模能力。该方法标志着从符号分词向类人视觉阅读的转变,推动更自然、更贴近认知的语言模型发展。

原文摘要 · Abstract (English)

People see text. Humans read by recognizing words as visual objects, including their shapes, layouts, and patterns, before connecting them to meaning, which enables us to handle typos, distorted fonts, and various scripts effectively. Modern large language models (LLMs), however, rely on subword tokenization, fragmenting text into pieces from a fixed vocabulary. While effective for high-resource languages, this approach over-segments low-resource languages, yielding long, linguistically meaningless sequences and inflating computation. In this work, we challenge this entrenched paradigm and move toward a vision-centric alternative. Our method, SeeTok, renders text as images (visual-text) and leverages pretrained multimodal LLMs to interpret them, reusing strong OCR and text-vision alignment abilities learned from large-scale multimodal training. Across three different language tasks, SeeTok matches or surpasses subword tokenizers while requiring 4.43 times fewer tokens and reducing FLOPs by 70.5%, with additional gains in cross-lingual generalization, robustness to typographic noise, and linguistic hierarchy. SeeTok signals a shift from symbolic tokenization to human-like visual reading, and takes a step toward more natural and cognitively inspired language models.

视觉阅读多模态分词革新语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。