arXiv:2506.06962cs.CV2025-06NeurIPS被引 7

让图像生成动态检索参考图,避免照搬和风格偏移。

AR-RAG: Autoregressive Retrieval Augmentation for Image Generation

  • 每步生成时用已生成局部图检索最相关参考块
  • 在多个基准上超越当前最优模型,显著提升质量
  • 无需训练的插件式设计,适合快速集成到现有模型

我们提出自回归检索增强(AR-RAG),一种在图像生成过程中逐块动态引入最近邻检索的新范式。与以往仅单次静态检索并固定依赖参考图的方法不同,AR-RAG在每一步生成中基于已生成的局部块作为查询,主动获取最相关的视觉参考块,从而适应生成过程中的动态需求,并缓解过量复制、风格偏差等问题。为实现该机制,我们提出两种并行框架:(1) 解码阶段分布增强(DAiD),一种无需训练的即插即用解码策略,直接融合模型预测块与检索块的概率分布;(2) 解码阶段特征增强(FAiD),一种参数高效微调方法,通过多尺度卷积逐步平滑检索块特征,并用于增强生成过程。我们在Midjourney-30K、GenEval和DPG-Bench等广泛采用的基准上验证了AR-RAG的有效性,结果表明其在性能上显著优于当前最先进的图像生成模型。

原文摘要 · Abstract (English)

We introduce Autoregressive Retrieval Augmentation (AR-RAG), a novel paradigm that enhances image generation by autoregressively incorporating knearest neighbor retrievals at the patch level. Unlike prior methods that perform a single, static retrieval before generation and condition the entire generation on fixed reference images, AR-RAG performs context-aware retrievals at each generation step, using prior-generated patches as queries to retrieve and incorporate the most relevant patch-level visual references, enabling the model to respond to evolving generation needs while avoiding limitations (e.g., over-copying, stylistic bias, etc.) prevalent in existing methods. To realize AR-RAG, we propose two parallel frameworks: (1) Distribution-Augmentation in Decoding (DAiD), a training-free plug-and-use decoding strategy that directly merges the distribution of model-predicted patches with the distribution of retrieved patches, and (2) Feature-Augmentation in Decoding (FAiD), a parameter-efficient fine-tuning method that progressively smooths the features of retrieved patches via multi-scale convolution operations and leverages them to augment the image generation process. We validate the effectiveness of AR-RAG on widely adopted benchmarks, including Midjourney-30K, GenEval and DPG-Bench, demonstrating significant performance gains over state-of-the-art image generation models.

图像生成检索增强自回归扩散模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。