用大模型拆解复杂指令,让文生图更准确。
DeCoT: Decomposing Complex Instructions for Enhanced Text-to-Image Generation with Large Language Models
- 用大模型把复杂指令分解成可执行的语义单元
- 在LongBench-T2I上使生成效果平均提升0.08分
- 适合需要精准还原细节和布局的用户
当前文生图模型在处理长文本、复杂指令时表现不佳,常无法准确生成细节、空间关系或特定约束。为解决此问题,本文提出DeCoT(分解思维链)框架,利用大语言模型增强文生图模型对复杂指令的理解与执行能力。DeCoT包含两个阶段:首先由大模型将原始指令分解为结构化、可操作的语义单元并消除歧义;其次将这些单元融合为分层或优化后的单个提示词,适配现有文生图模型。在LongBench-T2I数据集上的实验表明,DeCoT显著提升主流文生图模型性能,尤其在“文字”和“构图”维度。量化评估显示,结合Infinity-8B时,平均得分达3.52,优于基线模型的3.44。消融实验验证各组件有效性,人类评估也确认生成图像在感知质量与指令忠实度上更优。DeCoT有效弥合用户意图与文生图模型需求间的鸿沟。
原文摘要 · Abstract (English)
Despite remarkable advancements, current Text-to-Image (T2I) models struggle with complex, long-form textual instructions, frequently failing to accurately render intricate details, spatial relationships, or specific constraints. This limitation is highlighted by benchmarks such as LongBench-T2I, which reveal deficiencies in handling composition, specific text, and fine textures. To address this, we propose DeCoT (Decomposition-CoT), a novel framework that leverages Large Language Models (LLMs) to significantly enhance T2I models' understanding and execution of complex instructions. DeCoT operates in two core stages: first, Complex Instruction Decomposition and Semantic Enhancement, where an LLM breaks down raw instructions into structured, actionable semantic units and clarifies ambiguities; second, Multi-Stage Prompt Integration and Adaptive Generation, which transforms these units into a hierarchical or optimized single prompt tailored for existing T2I models. Extensive experiments on the LongBench-T2I dataset demonstrate that DeCoT consistently and substantially improves the performance of leading T2I models across all evaluated dimensions, particularly in challenging aspects like "Text" and "Composition". Quantitative results, validated by multiple MLLM evaluators (Gemini-2.0-Flash and InternVL3-78B), show that DeCoT, when integrated with Infinity-8B, achieves an average score of 3.52, outperforming the baseline Infinity-8B (3.44). Ablation studies confirm the critical contribution of each DeCoT component and the importance of sophisticated LLM prompting. Furthermore, human evaluations corroborate these findings, indicating superior perceptual quality and instruction fidelity. DeCoT effectively bridges the gap between high-level user intent and T2I model requirements, leading to more faithful and accurate image generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。