让AI画画时能查网页，自动补知识，画得更准更像。
World-To-Image: Grounding Text-to-Image Generation with Agent-Driven World Knowledge
- 用智能代理上网搜图，补足模型没学过的概念知识。
- 在自建数据集上比顶尖方法准确率提升8.1%。
- 适合需要画新事物、追求真实感的创意工作者。
尽管文本到图像（T2I）模型能生成高质量图像，但当提示包含新颖或分布外（OOD）实体时，其性能会因固有知识限制而显著下降。我们提出World-To-Image框架，通过引入由智能体驱动的世界知识，弥合这一差距。设计一个智能体动态检索基础模型未知概念的网络图片，再用于多模态提示优化，引导生成模型更精准合成图像。关键在于，评估不仅依赖传统指标，还采用LLMGrader和ImageReward等现代评测方式衡量真实语义一致性。实验表明，World-To-Image在语义对齐与视觉美感上均显著优于现有方法，在自建的NICE基准上实现提示准确率+8.1%的提升。该框架仅需不到三轮迭代即可达成效果，为能实时反映现实世界变化的T2I系统铺平道路。演示代码已公开于https://github.com/mhson-kyle/World-To-Image。
原文摘要 · Abstract (English)
While text-to-image (T2I) models can synthesize high-quality images, their performance degrades significantly when prompted with novel or out-of-distribution (OOD) entities due to inherent knowledge cutoffs. We introduce World-To-Image, a novel framework that bridges this gap by empowering T2I generation with agent-driven world knowledge. We design an agent that dynamically searches the web to retrieve images for concepts unknown to the base model. This information is then used to perform multimodal prompt optimization, steering powerful generative backbones toward an accurate synthesis. Critically, our evaluation goes beyond traditional metrics, utilizing modern assessments like LLMGrader and ImageReward to measure true semantic fidelity. Our experiments show that World-To-Image substantially outperforms state-of-the-art methods in both semantic alignment and visual aesthetics, achieving +8.1% improvement in accuracy-to-prompt on our curated NICE benchmark. Our framework achieves these results with high efficiency in less than three iterations, paving the way for T2I systems that can better reflect the ever-changing real world. Our demo code is available here\footnote{https://github.com/mhson-kyle/World-To-Image}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。