arXiv:2608.18679cs.CV2026-08

用分层图结构生成与编辑文档,让内容和布局协同演化。

FRAGMENT: Factorized Graph Representations for Document Generation and Editing via Entity-Aware Transformations

  • 将文档拆解为结构与内容的因子化图模型,分两阶段生成
  • 在三个数据集上生成质量优于基线,支持语义感知编辑
  • 适合需要精准布局控制的文档自动化场景

像发票、报告等结构化文档的意义来自空间布局、文本内容和逻辑结构的交互。现有像素或词元级生成模型难以有效捕捉这些依赖关系。本文提出FRAGMENT框架,将文档表示为带类型的关联图,并将分布因子化为p(结构, 内容) = p(结构) × p(内容|结构)。该框架包含两个阶段:第一阶段的Architect是因果掩码Transformer,基于文档类别自回归生成图拓扑和类型化空间关系;第二阶段的Builder是基于GATv2的图注意力网络,为图注入归一化边界框、文本及视觉风格属性。两阶段均定义显式似然模型,得到可计算的文档级似然,可用于伪造检测。针对可控编辑,通过交叉注意力注入指令嵌入,实现语义与实体感知修改。在DocLayNet上训练,在FUNSD和SROIE上微调。实验在DocLayNet、FUNSD和SROIE上对比了代表性自回归、仅布局和图基基线,提供了对所提因子化图生成框架特性与权衡的实证分析。

原文摘要 · Abstract (English)

Structured documents such as invoices, forms, reports, and scientific articles derive meaning from the interplay between spatial layout, textual content, and logical structure. Generative models operating at the pixel or token level often struggle to capture these dependencies effectively. We explore FRAGMENT, a generative framework that represents a document as a typed relational graph and factorizes its distribution as p(structure, content) = p(structure) * p(content | structure). The framework consists of two stages. The first stage, the Architect, is a causally masked Transformer conditioned on document category that autoregressively generates the graph topology and typed spatial relations. The second stage, the Builder, is a GATv2-based graph attention network that enriches the graph with normalized bounding boxes, text, and visual style attributes. Both stages define explicit likelihood models, yielding a tractable document-level likelihood that serves as an anomaly score for forgery detection. For controlled editing, a prompt-conditioned extension injects instruction embeddings into the Builder through cross-attention, enabling semantic and entity-aware modifications. We describe training on DocLayNet and fine-tuning on FUNSD and SROIE. Experiments on DocLayNet, FUNSD, and SROIE evaluate FRAGMENT alongside representative autoregressive, layout-only, and graph-based baselines, providing an empirical analysis of the characteristics and trade-offs of the proposed factorized graph generation framework.

文档生成图神经网络布局建模可控编辑

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。