构建400万级图文对数据集,提升文本生成的布局准确性
TextGround4M: A Prompt-Aligned Dataset for Layout-Aware Text Rendering

- 构建细粒度标注的文本布局数据集,支持跨段落精准定位
- 训练后模型在文本保真度与空间位置准确率上超越基线
- 适合需要高精度文本排版的图像生成研究者使用
尽管文本到图像生成取得进展,模型仍难以在多段落、结构化场景中准确渲染符合提示的文本布局。这不仅源于缺乏将提示与图像中精确文本及布局对齐的数据集,也因缺少有效的布局质量评估指标。为此,我们提出TextGround4M,一个包含超过400万组提示-图像对的大规模数据集,每对均标注了文本片段在提示中的对应位置及边界框,实现细粒度布局监督。基于此,我们设计一种轻量级训练策略,在自回归文本到图像模型训练中追加布局感知的片段标记,不改变模型架构或推理行为。同时,构建分层复杂度的基准测试集,用于零样本评估开源与专有模型。此外,提出两种布局感知评估指标,填补文本生成中空间评估长期缺失的空白。实验表明,使用TextGround4M训练的模型在文本保真度、空间准确性与提示一致性上均优于强基线,凸显细粒度布局监督对提示对齐图像生成的重要性。
原文摘要 · Abstract (English)
Despite recent advances in text-to-image generation, models still struggle to accurately render prompt-specified text with correct spatial layout -- especially in multi-span, structured settings. This challenge is driven not only by the lack of datasets that align prompts with the exact text and layout expected in the image, but also by the absence of effective metrics for evaluating layout quality. To address these issues, we introduce TextGround4M, a large-scale dataset of over 4 million prompt-image pairs, each annotated with span-level text grounded in the prompt and corresponding bounding boxes. This enables fine-grained supervision for layout-aware, prompt-grounded text rendering. Building on this, we propose a lightweight training strategy for autoregressive T2I models that appends layout-aware span tokens during training, without altering model architecture or inference behavior. We further construct a benchmark with stratified layout complexity to evaluate both open-source and proprietary models in a zero-shot setting. In addition, we introduce two layout-aware metrics to address the long-standing lack of spatial evaluation in text rendering. Our results show that models trained on TextGround4M outperform strong baselines in text fidelity, spatial accuracy, and prompt consistency, highlighting the importance of fine-grained layout supervision for grounded T2I generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。