提出一种新方法，让离散变量在模型训练中更高效地计算梯度。
Categorical Reparameterization with Denoising Diffusion models
- 用扩散模型构建连续松弛，实现可重参数化的梯度估计
- 在多个模型上表现优于或相当于现有方法，提升训练稳定性
- 适合需要优化离散变量的生成模型与强化学习场景
学习包含离散变量的模型需要对离散分布的期望进行优化，但离散采样不可导，使得基于随机梯度的优化困难。常见做法是将离散分布替换为连续松弛，从而获得平滑近似，支持通过重参数化技巧计算梯度。本文在此基础上提出 ReDGE——一种基于扩散模型的新型高效软重参数化方法。该方法定义了一类灵活的梯度估计器，其中包含直通估计器（Straight-Through）作为特例。实验涵盖隐变量模型及离散扩散模型中的推理时奖励引导，结果表明 ReDGE 在各项任务中均稳定达到或超越现有基于梯度的方法。代码将发布于 https://github.com/samsongourevitch/redge。
原文摘要 · Abstract (English)
Learning models with categorical variables requires optimizing expectations over discrete distributions, a setting in which stochastic gradient-based optimization is challenging due to the non-differentiability of categorical sampling. A common workaround is to replace the discrete distribution with a continuous relaxation, yielding a smooth surrogate that admits reparameterized gradient estimates via the reparameterization trick. Building on this idea, we introduce ReDGE, a novel and efficient diffusion-based soft reparameterization method for categorical distributions. Our approach defines a flexible class of gradient estimators that includes the Straight-Through estimator as a special case. Experiments spanning latent variable models and inference-time reward guidance in discrete diffusion models demonstrate that ReDGE consistently matches or outperforms existing gradient-based methods. The code will be made available at https://github.com/samsongourevitch/redge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。