用MCMC统一渲染、优化与生成AI,打通物理真实图像生成路径
MCMC: Bridging Rendering, Optimization and Generative AI
- 以MCMC为桥梁,连接生成模型采样、梯度优化与物理渲染
- 揭示梯度更新与马尔可夫链的内在一致性,支持无记忆参数空间探索
- 适合想理解生成模型底层机制的研究者与实践者
生成式人工智能在过去两年中在视觉语言模型方面取得了前所未有的进展。生成过程中,新样本(图像)从一个未知的高维分布中生成。马尔可夫链蒙特卡洛(MCMC)方法在从这类复杂高维分布中抽样方面尤为有效,使其成为能量基模型(EBMs)等模型中确保准确采样的关键组件。基于梯度的优化是现代生成模型的核心,优化过程中的更新步骤构成一个马尔可夫链,新更新仅依赖于当前状态,从而实现无记忆的参数空间探索,结合了梯度优化与MCMC采样的优势。MCMC方法在基于物理的渲染中同样至关重要,因其能有效处理难以通过简单重要性采样获得的复杂光路。尽管大量研究致力于以数据驱动方式提升扩散生成模型的物理真实性,但连接这些技术的统一框架仍不明确。本课程首次尝试深入理解各组成部分,并探讨MCMC作为潜在桥梁,连接这些密切相关领域的可能性。课程旨在为学生、研究人员和从业者提供必要的理论与实践工具,推动生成式物理渲染的共同目标。所有配套的Jupyter笔记本演示均可在项目网页:https://sinbag.github.io/mcmc/ 获取。
原文摘要 · Abstract (English)
Generative artificial intelligence (AI) has made unprecedented advances in vision language models over the past two years. During the generative process, new samples (images) are generated from an unknown high-dimensional distribution. Markov Chain Monte Carlo (MCMC) methods are particularly effective in drawing samples from such complex, high-dimensional distributions. This makes MCMC methods an integral component for models like EBMs, ensuring accurate sample generation. Gradient-based optimization is at the core of modern generative models. The update step during the optimization forms a Markov chain where the new update depends only on the current state. This allows exploration of the parameter space in a memoryless manner, thus combining the benefits of gradient-based optimization and MCMC sampling. MCMC methods have shown an equally important role in physically based rendering where complex light paths are otherwise quite challenging to sample from simple importance sampling techniques. A lot of research is dedicated towards bringing physical realism to samples (images) generated from diffusion-based generative models in a data-driven manner, however, a unified framework connecting these techniques is still missing. In this course, we take the first steps toward understanding each of these components and exploring how MCMC could potentially serve as a bridge, linking these closely related areas of research. Our course aims to provide necessary theoretical and practical tools to guide students, researchers and practitioners towards the common goal of generative physically based rendering. All Jupyter notebooks with demonstrations associated to this tutorial can be found on the project webpage: https://sinbag.github.io/mcmc/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。