arXiv:2606.16323cs.CVcs.GR2026-06

用多层自适应融合提升单图人体材质估计精度

HAFMat: Hybrid Priors Guided Adaptive Fusion for Single-Image Human Material Estimation

论文配图:HAFMat: Hybrid Priors Guided Adaptive Fusion for Single-Image Human Material Estimation
图 1 · 摘自论文原文
  • 引入外观、几何、结构和预训练先验的引导图
  • 多层自适应融合机制使纹理与语义线索各得其所
  • 在合成与真实数据上均达到当前最优效果

基于物理的渲染(PBR)材质估计是虚拟内容创作、重光照和数字人渲染中的基础外观分解任务。然而,从单张人体图像中估计PBR材质仍极具挑战性,因为光照、几何与反射率在观测外观中高度耦合。为此,我们提出HAFMat,一种基于混合先验引导的单图人体材质估计框架。方法引入编码互补线索的引导图,包括外观、身体几何、结构以及来自预训练模型的先验材质预测。关键观察是这些引导线索具有异质性:部分提供纹理级约束,另一些传递高层语义信息。为此,设计了多层自适应特征融合机制,在不同阶段自适应融合引导特征与解码器特征,使纹理主导与语义主导线索在适当层级引导材质解码,从而实现更准确且物理合理的材质估计。在合成与真实数据上的大量实验表明,本方法在材质估计及下游重光照任务中均达到当前最优性能。

原文摘要 · Abstract (English)

Physically based rendering (PBR) material estimation is a fundamental appearance decomposition task with broad applications in virtual content creation, relighting, and digital human rendering. However, estimating PBR materials from a single human image remains highly ill-posed, since illumination, geometry, and reflectance are heavily entangled in the observed appearance. To mitigate this ambiguity, we propose HAFMat, a hybrid-prior-guided framework for single-image human material estimation. Our method introduces guidance maps that encode complementary cues, including appearance, body geometry, structure, and prior material predictions from pre-trained models. A key observation is that these guidance cues are heterogeneous: some cues mainly provide texture-level constraints, while others convey higher-level semantic information. To exploit this property, we design a Multi-layer Adaptive Feature Fusion Mechanism, which adaptively fuses guidance features with decoder features at different stages. This design enables texture-dominant and semantic-dominant cues to guide material decoding at appropriate levels, leading to more accurate and physically plausible material estimation. Extensive experiments on both synthetic and real data demonstrate that our method achieves state-of-the-art performance in material estimation and downstream relighting.

材质估计单图重建自适应融合数字人

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。