用真实数据提升单图3D浮雕恢复精度与鲁棒性
MonoRelief V2: Leveraging Real Data for High-Fidelity Monocular Relief Recovery
- 结合伪真实图像与真实数据，分阶段训练提升模型泛化能力
- 在800个真实样本上实现超越前代的深度与法向预测效果
- 适合需要高保真浮雕重建的应用场景，如数字艺术与文化遗产
本文提出MonoRelief V2，一种端到端模型，可在复杂材质和光照条件下直接从单张图像恢复2.5D浮雕。相比仅依赖合成数据的V1版本，V2引入真实数据以增强鲁棒性、准确性和效率。为克服大规模真实数据获取难题，研究者利用文生图模型生成约15,000张伪真实图像，并通过深度与法向预测融合获得伪标签；同时构建包含800个样本的小规模真实数据集，基于多视角重建与细节优化。模型在伪真实与真实数据上逐步训练，实验表明其在深度与法向预测上均达到当前最优性能，展现出广泛下游应用潜力。代码已开源。
原文摘要 · Abstract (English)
This paper presents MonoRelief V2, an end-to-end model designed for directly recovering 2.5D reliefs from single images under complex material and illumination variations. In contrast to its predecessor, MonoRelief V1 [1], which was solely trained on synthetic data, MonoRelief V2 incorporates real data to achieve improved robustness, accuracy and efficiency. To overcome the challenge of acquiring large-scale real-world dataset, we generate approximately 15,000 pseudo real images using a text-to-image generative model, and derive corresponding depth pseudo-labels through fusion of depth and normal predictions. Furthermore, we construct a small-scale real-world dataset (800 samples) via multi-view reconstruction and detail refinement. MonoRelief V2 is then progressively trained on the pseudo-real and real-world datasets. Comprehensive experiments demonstrate its state-of-the-art performance both in depth and normal predictions, highlighting its strong potential for a range of downstream applications. Code is at: https://github.com/glp1001/MonoreliefV2.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。