让图像生成模型从过往经验中学习,提升未来表现
MemoGen: Can Past Experience Improve Future Text-to-Image Generation?

- 通过构建经验记忆库,记录每次生成的成败与策略
- 仅用两次演化轮次就超越多个商用系统性能
- 无需更新模型参数,适合长期部署的生成系统
当前文本到图像模型在处理隐含视觉约束、关系推理或外部知识的需求时仍不可靠。现有方法虽借助外部知识或提示优化,但多将每次生成视为独立事件,未系统保留过往经验。本文提出MemoGen,一个无需训练的框架,在不更新生成器的前提下,通过引入代理演化层,为每个任务显式推断视觉需求,必要时检索外部证据与参考,将其转化为可执行生成约束,评估结果,并将任务理解、参考选择、视觉反馈、成功策略与失败教训存入可复用的经验记忆。在后续生成中,代理会检索相关经验以改进相似任务,选择性修复失败案例并保留成功模式,实现测试时自进化。在知识密集型与推理导向的基准上,仅经两轮演化,基于Qwen-Image的MemoGen即超越Nano Banana Pro与GPT-Image-1等强商业系统,证明显式经验记忆可成为可靠生成的强大持续学习信号。
原文摘要 · Abstract (English)
Modern text-to-image models have achieved strong visual synthesis, yet remain unreliable when prompts require implicit visual constraints, relational reasoning, or external knowledge. Existing retrieval-augmented and agentic generation methods mitigate this issue by acquiring external knowledge, references, or refined prompts for the current request, yet they typically treat each generation as an isolated episode and do not systematically preserve past successes or failures for future use. In this work, we ask whether a text-to-image system can continually improve from its own generation experience without updating the underlying generator. We propose MemoGen, a training-free framework that augments existing image generators with an agentic evolution layer. For each task, MemoGen explicitly infers visual requirements, retrieves external evidence and references when necessary, translates them into executable generation constraints, evaluates the generated result, and stores task understanding, reference choices, visual feedback, successful strategies, and failure lessons as reusable experience memory. Across evolution rounds, the agent retrieves relevant experience to improve similar future generations, selectively repairing previously failed cases while preserving successful ones, thereby enabling test-time self-evolution without parameter updates. Extensive experiments on knowledge-intensive and reasoning-oriented benchmarks demonstrate the effectiveness of this paradigm: after only two evolution rounds, MemoGen built upon the open-source Qwen-Image backbone surpasses strong proprietary systems such as Nano Banana Pro and GPT-Image-1 on WISE and Mind-Bench, showing that explicit experience memory can serve as a powerful continual learning signal for reliable text-to-image generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。