用长结构化描述训练图像生成模型,提升精准控制能力。
Generating an Image From 1,000 Words: Enhancing Text-to-Image With Structured Captions
- 用细粒度属性标注的长文本训练模型,实现视觉要素解耦控制。
- 提出新评估方法TaBR,可量化长文本生成的可控性与表达力。
- 开源模型FIBO在开放模型中达到最优提示对齐效果,适合专业创作。
文本到图像模型已从随意创作工具演进为专业级系统,实现了前所未有的图像质量和真实感。然而,多数模型仅能将简短提示映射为细节丰富的图像,导致文本输入稀疏与视觉输出丰富之间的不匹配,降低可控性,模型常任意填补缺失信息,偏向平均用户偏好,限制专业应用。本文首次在长结构化描述上训练开源文本到图像模型,每个训练样本均标注相同的一组细粒度属性,最大化表达覆盖并支持视觉因素的解耦控制。为高效处理长文本,提出轻量级模型融合机制DimFusion,无需增加词元长度即可整合中间特征。同时引入文本作为瓶颈重建(TaBR)评估协议,通过图像-描述-生成循环重构评估,直接衡量长文本下的可控性与表达力,克服现有方法在长文本场景下的失效问题。最终基于这些贡献训练出大规模模型FIBO,其在开源模型中实现最佳提示对齐表现。模型权重已公开于https://huggingface.co/briaai/FIBO。
原文摘要 · Abstract (English)
Text-to-image models have rapidly evolved from casual creative tools to professional-grade systems, achieving unprecedented levels of image quality and realism. Yet, most models are trained to map short prompts into detailed images, creating a gap between sparse textual input and rich visual outputs. This mismatch reduces controllability, as models often fill in missing details arbitrarily, biasing toward average user preferences and limiting precision for professional use. We address this limitation by training the first open-source text-to-image model on long structured captions, where every training sample is annotated with the same set of fine-grained attributes. This design maximizes expressive coverage and enables disentangled control over visual factors. To process long captions efficiently, we propose DimFusion, a fusion mechanism that integrates intermediate tokens from a lightweight LLM without increasing token length. We also introduce the Text-as-a-Bottleneck Reconstruction (TaBR) evaluation protocol. By assessing how well real images can be reconstructed through a captioning-generation loop, TaBR directly measures controllability and expressiveness, even for very long captions where existing evaluation methods fail. Finally, we demonstrate our contributions by training the large-scale model FIBO, achieving state-of-the-art prompt alignment among open-source models. Model weights are publicly available at https://huggingface.co/briaai/FIBO
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。