用扩散模型实现3D物体材质的高保真重建，支持任意视角输入。
MatMart: Material Reconstruction of 3D Objects via Diffusion
- 两阶段重建：先精准预测材质，再生成缺失视角的材质。
- 支持任意数量输入图像，通过渐进推理与交叉注意力提升效果。
- 单模型端到端训练，无需预训练模型，稳定适应多种物体类型。
将扩散模型应用于基于物理的材质估计与生成近年来备受关注。本文提出 tt，一种面向3D物体的新型材质重建框架，具有以下优势：首先，采用两阶段重建策略，先从输入中准确预测材质，再通过先验引导生成未观测视角的材质，实现高保真输出；其次，结合渐进推理与提出的视图-材质交叉注意力（VMCA），可从任意数量输入图像中进行重建，展现出强可扩展性与灵活性；最后，通过单一扩散模型的端到端优化，同时实现材质预测与生成，无需依赖额外预训练模型，从而在各类物体上表现出更强的稳定性。大量实验表明， tt 在材质重建性能上优于现有方法。
原文摘要 · Abstract (English)
Applying diffusion models to physically-based material estimation and generation has recently gained prominence. In this paper, we propose \ttt, a novel material reconstruction framework for 3D objects, offering the following advantages. First, \ttt\ adopts a two-stage reconstruction, starting with accurate material prediction from inputs and followed by prior-guided material generation for unobserved views, yielding high-fidelity results. Second, by utilizing progressive inference alongside the proposed view-material cross-attention (VMCA), \ttt\ enables reconstruction from an arbitrary number of input images, demonstrating strong scalability and flexibility. Finally, \ttt\ achieves both material prediction and generation capabilities through end-to-end optimization of a single diffusion model, without relying on additional pre-trained models, thereby exhibiting enhanced stability across various types of objects. Extensive experiments demonstrate that \ttt\ achieves superior performance in material reconstruction compared to existing methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。