无需梯度计算即可高效控制扩散模型生成，速度提升显著。
General and Efficient Steering of Diffusion Models
- 结合噪声对齐与递归特征机，离线计算控制信号
- 在CIFAR-10等数据集上准确率优于基于梯度的方法
- 推理速度大幅加快，适合需要快速生成的场景
将扩散模型引导至训练中未见的条件通常需微调或每步计算梯度，带来巨大开销。本文提出噪声对齐的RFM控制方法（NA-RFM），无需推理时梯度即可高效控制生成。该方法离线计算两个信号：基于目标样本与全数据主成分分析的噪声对齐，以及从标注前向过程激活中学习的目标判别方向。采样时，噪声对齐在高噪声阶段提供粗粒度控制，而RFM方向通过轻量激活修改在中后期重复使用。在CIFAR-10、ImageNet、CelebA及鸟类细粒度分类任务上，相比基于梯度的后处理引导方法，目标准确率更高，类别引导基准下FID更优，且推理速度显著提升。代码已开源。
原文摘要 · Abstract (English)
Steering diffusion models toward conditions unseen during training typically requires either retraining with conditional inputs or per-step gradient computations, both of which incur substantial computational overhead. We present Noise-Aligned RFM Steering (NA-RFM), a general recipe for efficiently steering diffusion models without gradient guidance during inference, enabling fast controllable generation. The method combines two offline-computed signals: noise alignment, a high-noise correction from PCA statistics of the target examples and the full data, and Recursive Feature Machine (RFM) activation steering, which learns a target-discriminative direction from labeled forward-process activations. During sampling, noise alignment provides coarse control at high noise, while the RFM direction is reused over intermediate/late timesteps through lightweight activation edits. Experiments on CIFAR-10, ImageNet, CelebA, and fine-grained bird species show improved target accuracy over gradient-based post-hoc guidance baselines, improved FID on the class-guidance benchmarks, and substantial inference speedups. Code: https://github.com/isotrivial/na-rfm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。