提出新方法解决3D生成中能量匹配的结构与计算难题
Projected Energy Matching for Generative 3D Priors

- 用哈密顿蒸馏将流模型中的旋转噪声引入残差网络
- 通过负样本缓存实现高效对比训练,降低计算开销
- 在医学CT稀疏重建中实现高质量恢复,适合医疗图像生成
能量匹配作为一种生成框架,结合了流模型的效率与能量基模型的显式似然,仅依赖单一时间无关标量势能。然而,在高维3D数据上直接训练该势能仍存在计算挑战。虽然蒸馏预训练流模型可降低初始训练成本,但其速度场不可避免包含非保守旋转伪影(旋度)。强制一个严格保守的标量势能去匹配这种无约束场会产生“结构冲突”,导致生成质量下降和模式覆盖不足。为此,我们提出投影能量匹配,一种可扩展的框架,解决了结构与计算瓶颈。我们引入哈密顿蒸馏,利用霍奇森迹估计器,将旋转噪声显式吸收至辅助残差网络。随后通过负样本缓存策略,在梯度累积的对比训练中复用负样本,使采样变得可行。我们将该方法作为真实世界医学CT逆问题的无条件先验,应用于稀疏视图重建。最终,我们的近似化流水线将总计算量减少至标准能量匹配的小部分,同时实现高保真重建,并成功消除严重测量伪影。
原文摘要 · Abstract (English)
Energy Matching has emerged as a powerful generative framework that combines flow model efficiency with the explicit likelihood of Energy-Based Models (EBMs) via a single, time-independent scalar potential. However, directly training this potential on high-dimensional 3D data remains computationally challenging. While distilling a pre-trained flow model circumvents some of the initial training costs, we demonstrate that velocity fields inevitably contain non-conservative rotational artifacts (curl). Forcing a strictly conservative scalar potential to match this unconstrained field creates a "structural conflict", which degrades generation quality and mode coverage. To solve this, we propose Projected Energy Matching, a scalable framework that resolves these structural and computational bottlenecks. We introduce Helmholtz Distillation, a structural relaxation that leverages a Hutchinson trace estimator to explicitly absorb rotational noise into an auxiliary residual network. We subsequently refine this landscape using Negative Caching, a memory-efficient strategy that reuses negative samples across micro-batches, rendering sampling tractable during contrastive training with gradient accumulation. We deploy our method as an unconditional prior for real-world medical CT inverse problems, specifically sparse-view reconstruction. Ultimately, our amortized pipeline reduces total compute to a small fraction of that required by standard energy matching, while achieving high-fidelity reconstructions and successfully resolving severe measurement artifacts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。