arXiv:2604.27272cs.CLcs.AI2026-04

将二维结构任务转为一维文本会丢失空间关系,影响模型性能。

When 2D Tasks Meet 1D Serialization: On Serialization Friction in Structured Tasks

  • 用一维文本序列化二维布局任务,导致空间关系隐含
  • 任务规模增大时,一维序列化错误率显著上升
  • 适合研究视觉布局依赖任务的模型表现差异

在大模型时代,许多符号性和结构性问题通过一维文本序列化呈现给模型。然而,部分问题本质上是二维的:其相关关系(如行列对应、空间邻接)由二维布局位置定义,而非顺序。这引发一个表示性问题:相同的符号内容在一维序列中是否仍能保留计算所需的结构关系?本文通过‘序列化摩擦’视角研究该问题——相同任务实例和内容下,依赖布局的关系在序列化后变得隐式。实验使用三个可控合成任务:矩阵转置、康威生命游戏和LU分解。同一任务以一维文本或原始二维图像形式呈现。结果表明,随着任务规模增加,一维序列化性能下降更剧烈,且错误呈现空间模式,说明表示方式有显著影响。补充分析包括视觉探针和混合训练转置任务对比,进一步揭示对于布局定义的任务,简化为一维序列并非中性选择。

原文摘要 · Abstract (English)

In the LLM era, many symbolic and structured problems are presented to models through 1D text serialization. Yet some such problems are natively two-dimensional: their relevant relations, such as row--column correspondence or spatial adjacency, are defined by position in a 2D layout rather than by sequential order. This raises a representational question: does preserving the same symbolic entries in a 1D sequence also preserve the relational structure needed for computation? We study this issue through the lens of serialization friction: the representational mismatch in which the same underlying task instances and entries are still present, but relations that depend on layout become implicit under 1D serialization. The study uses a controlled synthetic testbed of three tasks: matrix transpose, Conway's Game of Life, and LU decomposition. In each task, the same instances are presented either as 1D text serialization or as their native 2D layout rendered as an image. Across this testbed, 1D serialization degrades more sharply as task size grows, and errors under serialization exhibit spatially structured patterns, suggesting that this presentation choice is consequential within our testbed. To further interpret these results, we add supplementary analyses that include a within-visual probe and an additional comparison of the two input presentations under the mixed-training transpose setting. These findings suggest that, for layout-defined tasks, reducing inputs to 1D serialization is not a neutral choice of representation.

序列化摩擦二维任务布局感知模型输入

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。