降低扩散模型梯度估算方差，提升生成任务效率
Variance Reduction for Expectations with Diffusion Teachers
- 用分层蒙特卡洛估计，复用昂贵上游计算
- 文本到3D任务中有效算力提升2-3倍
- 适合需要高效梯度计算的生成式建模场景
预训练扩散模型作为固定教师，用于文本到3D、单步蒸馏和数据归因等下游任务。这些管道消耗的教师梯度是噪声水平和高斯噪声样本上的蒙特卡洛（MC）期望；其估计方差主导了计算成本，因为每次采样都需要昂贵的上游操作（渲染、仿真、编码）。我们提出CARV，一种计算感知的方差核算框架，驱动一种分层蒙特卡洛估计器：在廉价的扩散噪声重采样上摊薄昂贵的上游计算，通过时间步重要性采样和分层逆CDF构造进一步优化。在文本到3D蒸馏与归因实验中，CARV实现2-3倍的有效算力提升（主要来自复用；约25%额外增益来自重要性采样+分层）且不改变目标函数；在单步蒸馏中，相同技术将梯度方差降低一个数量级，但未改善下游FID，表明此时蒙特卡洛方差已不再是瓶颈。
原文摘要 · Abstract (English)
Pretrained diffusion models serve as frozen teachers feeding downstream pipelines such as text-to-3D, single-step distillation, and data attribution. The teacher gradients these pipelines consume are Monte Carlo (MC) expectations over noise levels and Gaussian noise samples; their estimator variance dominates compute cost because each draw requires expensive upstream work (rendering, simulation, encoding). We introduce CARV, a compute-aware variance-accounting framework that motivates a hierarchical MC estimator: amortize the expensive upstream computation over cheap diffusion-noise resamples, sharpened by timestep importance sampling and a stratified-inverse-CDF construction. In our text-to-3D distillation and attribution experiments, CARV delivers 2-3x effective compute multipliers (most from amortized reuse; ~25% additional from IS+stratification) without changing the objective; in single-step distillation, the same techniques cut gradient variance by an order of magnitude but do not improve downstream FID, marking the regime where MC variance is no longer the bottleneck.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。