arXiv:2502.01692cs.LGcs.AI2025-02ICLR被引 5

提出高效在线黑盒引导生成方法,无需训练数据或可微目标函数。

Fast Direct: Query-Efficient Online Black-box Guidance for Diffusion-model Target Generation

  • 构建数据流形上的伪目标,用统一方向更新扩散噪声序列
  • 图像生成任务提升6至10倍、分子生成提升11至44倍查询效率
  • 适合无现成数据集或不可微目标的真实场景,如人偏好图像生成

引导扩散模型生成是定制预训练扩散模型生成过程以应对特定下游任务的有前景方向。现有方法要么依赖预先收集的数据集训练引导模型,要么要求目标函数可微。然而,在多数真实场景中,离线数据集往往不可用,且目标函数通常不可微,例如基于人类偏好的图像生成、药物发现中的分子生成和材料设计。因此,需要一种在线算法,在运行时动态收集数据,并支持黑盒目标函数。此外,算法的查询效率至关重要,因为实际场景中目标评估代价高昂。本文提出一种新颖且简单的算法Fast Direct,用于高效在线黑盒目标生成。Fast Direct在数据流形上构建伪目标,通过通用方向更新扩散模型的噪声序列,实现高效的引导生成。在十二个高分辨率(1024×1024)图像目标生成任务和六个3D分子目标生成任务上的大量实验表明,查询效率提升了6至10倍(图像)和11至44倍(分子)。代码已公开:https://github.com/kimyong95/guide-stable-diffusion/tree/fast-direct。

原文摘要 · Abstract (English)

Guided diffusion-model generation is a promising direction for customizing the generation process of a pre-trained diffusion model to address specific downstream tasks. Existing guided diffusion models either rely on training the guidance model with pre-collected datasets or require the objective functions to be differentiable. However, for most real-world tasks, offline datasets are often unavailable, and their objective functions are often not differentiable, such as image generation with human preferences, molecular generation for drug discovery, and material design. Thus, we need an $\textbf{online}$ algorithm capable of collecting data during runtime and supporting a $\textbf{black-box}$ objective function. Moreover, the $\textbf{query efficiency}$ of the algorithm is also critical because the objective evaluation of the query is often expensive in real-world scenarios. In this work, we propose a novel and simple algorithm, $\textbf{Fast Direct}$, for query-efficient online black-box target generation. Our Fast Direct builds a pseudo-target on the data manifold to update the noise sequence of the diffusion model with a universal direction, which is promising to perform query-efficient guided generation. Extensive experiments on twelve high-resolution ($\small {1024 \times 1024}$) image target generation tasks and six 3D-molecule target generation tasks show $\textbf{6}\times$ up to $\textbf{10}\times$ query efficiency improvement and $\textbf{11}\times$ up to $\textbf{44}\times$ query efficiency improvement, respectively. Our implementation is publicly available at: https://github.com/kimyong95/guide-stable-diffusion/tree/fast-direct

扩散模型黑盒优化生成效率在线学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。