跨域材料检索新框架,提升真实感3D资产生成效果
MaRI: Material Retrieval Integration across Domains
- 通过对比学习构建合成与真实材料的共享嵌入空间
- 在复杂任务中优于现有方法,实现更高准确率与泛化能力
- 适合3D建模、游戏开发等需要高真实感材质的应用
精准的材料检索对生成逼真3D资产至关重要。现有方法依赖于捕捉形状不变但光照多变的材料数据集,这类数据稀缺且多样性不足,难以泛化到真实场景。多数方法采用传统图像搜索技术,无法有效捕获材料空间的独特属性,导致检索性能不佳。为此,我们提出MaRI框架,旨在弥合合成与真实材料之间的特征空间差距。MaRI通过联合训练图像编码器与材料编码器,采用对比学习策略构建共享嵌入空间,使相似材料及其图像在特征空间中更接近,而不同材料则被分离。为支持该框架,我们构建了一个综合性数据集,包含在受控形状变化和多样化光照条件下渲染的高质量合成材料,以及通过材料迁移技术处理并标准化的真实世界材料。大量实验表明,MaRI在多样且复杂的材料检索任务中展现出卓越的性能、准确性和泛化能力,显著优于现有方法。
原文摘要 · Abstract (English)
Accurate material retrieval is critical for creating realistic 3D assets. Existing methods rely on datasets that capture shape-invariant and lighting-varied representations of materials, which are scarce and face challenges due to limited diversity and inadequate real-world generalization. Most current approaches adopt traditional image search techniques. They fall short in capturing the unique properties of material spaces, leading to suboptimal performance in retrieval tasks. Addressing these challenges, we introduce MaRI, a framework designed to bridge the feature space gap between synthetic and real-world materials. MaRI constructs a shared embedding space that harmonizes visual and material attributes through a contrastive learning strategy by jointly training an image and a material encoder, bringing similar materials and images closer while separating dissimilar pairs within the feature space. To support this, we construct a comprehensive dataset comprising high-quality synthetic materials rendered with controlled shape variations and diverse lighting conditions, along with real-world materials processed and standardized using material transfer techniques. Extensive experiments demonstrate the superior performance, accuracy, and generalization capabilities of MaRI across diverse and complex material retrieval tasks, outperforming existing methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。