用2D扩散模型生成会开花的玫瑰等动态物体，无需人工建模。
Birth and Death of a Rose
- 从2D扩散模型中蒸馏信号，用神经模板保证时间一致性。
- 可生成高质动态物体序列，支持任意视角和光照下的渲染。
- 适合影视动画、虚拟现实等领域快速生成自然现象动画。
我们研究从预训练的2D基础模型生成时间性物体内在属性——如绽放的玫瑰这类随时间演变的物体几何、反射率与纹理——的问题。与传统3D建模和动画需大量手工操作不同，我们提出一种方法，通过从预训练2D扩散模型中提取信号来生成这些资产。为确保物体内在属性的时间一致性，我们提出了基于自监督学习图像特征自动推导出的神经模板，用于时态状态引导的蒸馏。该方法可生成多种自然现象的高质量时间序列物体属性，并支持在任意视角、任意环境光照条件及生命阶段下进行采样与可控渲染。项目网站：https://chen-geng.com/rose4d
原文摘要 · Abstract (English)
We study the problem of generating temporal object intrinsics -- temporally evolving sequences of object geometry, reflectance, and texture, such as a blooming rose -- from pre-trained 2D foundation models. Unlike conventional 3D modeling and animation techniques that require extensive manual effort and expertise, we introduce a method that generates such assets with signals distilled from pre-trained 2D diffusion models. To ensure the temporal consistency of object intrinsics, we propose Neural Templates for temporal-state-guided distillation, derived automatically from image features from self-supervised learning. Our method can generate high-quality temporal object intrinsics for several natural phenomena and enable the sampling and controllable rendering of these dynamic objects from any viewpoint, under any environmental lighting conditions, at any time of their lifespan. Project website: https://chen-geng.com/rose4d
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。