用7万张标注字型数据训练模型,让AI更准确理解文字风格和用途。
FontUse: A Data-Centric Approach to Style- and Use-Case-Conditioned In-Image Typography
- 构建7万张带字体风格与场景标注的图像数据集
- 模型生成文字更贴合用户指定的字体样式和使用场景
- 适合需要精准控制文本视觉效果的设计人员
当前文本到图像模型在生成高质量图像方面表现良好,但对文字排版的控制仍存在困难:用户指定的字体外观常被忽略或仅部分遵循。为此,我们提出一种以数据为中心的方法,通过针对字型设计的结构化标注流程,构建大规模字体专用数据集FontUse,包含约7万张图像,每张均标注了用户友好的提示词、文本区域位置及OCR识别出的字符串。这些标注利用分割模型和多模态大语言模型(MLLMs)自动生成。提示词明确结合字体风格(如衬线体、手写体、优雅风格)与使用场景(如婚礼请柬、咖啡馆菜单),使新手也能直观输入。在不修改架构的前提下,仅通过微调现有生成器即可使其稳定理解风格与场景条件。评估采用基于Long-CLIP的指标,衡量生成文字与请求属性的一致性。跨多种提示和布局的实验表明,经本方法训练的模型生成的文字更符合用户意图,优于现有基线。代码已开源于https://github.com/xiaxinz/FontUSE。
原文摘要 · Abstract (English)
Recent text-to-image models can generate high-quality images from natural-language prompts, yet controlling typography remains challenging: requested typographic appearance is often ignored or only weakly followed. We address this limitation with a data-centric approach that trains image generation models using targeted supervision derived from a structured annotation pipeline specialized for typography. Our pipeline constructs a large-scale typography-focused dataset, FontUse, consisting of about 70K images annotated with user-friendly prompts, text-region locations, and OCR-recognized strings. The annotations are automatically produced using segmentation models and multimodal large language models (MLLMs). The prompts explicitly combine font styles (e.g., serif, script, elegant) and use cases (e.g., wedding invitations, coffee-shop menus), enabling intuitive specification even for novice users. Fine-tuning existing generators with these annotations allows them to consistently interpret style and use-case conditions as textual prompts without architectural modification. For evaluation, we introduce a Long-CLIP-based metric that measures alignment between generated typography and requested attributes. Experiments across diverse prompts and layouts show that models trained with our pipeline produce text renderings more consistent with prompts than competitive baselines. The source code for our annotation pipeline is available at https://github.com/xiaxinz/FontUSE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。