用分枝推理提升图像生成的语义一致性
Tree-of-Thoughts Reasoning for Text-to-Image In-Context Learning
- 构建多阶段推理框架，探索多种解释并筛选最优方案
- 在CoBSAT上生成图像更一致且语义准确，无需额外训练
- 适合需要精准控制生成内容的研究者和开发者
在文本到图像的上下文学习（T2I-ICL）中，模型需从少量示例中推断出潜在的组合模式以生成目标图像。近期研究表明，当前先进的多模态大模型在此任务中表现不佳，主要由于组合推理能力有限及对提示构造敏感。本文提出一种面向T2I-ICL的思维树（ToT）推理框架，引入多阶段推理与选择层，先生成多个候选假设，再评估并选择最合理的解释，最终构建用于图像合成的提示。通过探索不同推理路径并选取连贯的解读，该方法有效缓解了提示歧义与组合错误。我们在完整ToT-T2IICL推理流程中实现该方法，并在CoBSAT基准上进行评估。定性和定量结果均表明，结构化多分支推理显著提升了图像生成的一致性与语义对齐程度，优于基线和链式思维提示策略，且无需任何额外训练或微调。
原文摘要 · Abstract (English)
In text-to-image in-context learning (T2I-ICL), a model has to infer a latent compositional pattern from fewshot demonstrations for generating a query image. Recent studies show that state-of-the-art multimodal large language models struggle with this setting, particularly due to limited compositional reasoning and sensitivity to prompt construction. In this work, we propose a Tree-of-Thoughts (ToT) reasoning framework for T2I-ICL that introduces a multi-stage reasoning and selection layer that generates, evaluates, and selects among multiple candidate hypotheses before constructing the final prompt for image synthesis. By exploring alternative reasoning branches and selecting a coherent interpretation, the proposed approach mitigates prompt ambiguity and compositional errors. We implement the proposed approach in a complete ToT-T2IICL inference pipeline and evaluate it on the CoBSAT benchmark. Both qualitative and quantitative results show that structured multi-branch reasoning leads to more consistent and semantically aligned image generation compared to baseline and Chain-of-Thought prompting strategies, without any additional training or fine-tuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。