从单目视频重建未知物体的高保真纹理，首次考虑手物交互影响。
TexHOI: Reconstructing Textures of 3D Unknown Objects in Monocular Hand-Object Interaction Scenes
- 通过辐射场合成学习物体、手和背景的几何与低质纹理
- 优化手物姿态并调整光照参数，提升材质与阴影精度
- 适合做真实场景三维物体建模的研究者和开发者
近年来，从单目图像序列中重建动态真实物体的高保真3D模型与纹理一直是个难题，主要源于阴影、间接光照以及手物遮挡导致的姿态估计不准。为此，我们提出一种新方法，能够预测手对环境可见性及物体表面反照率的间接光照影响。首先，通过辐射场的组合渲染学习物体、手和背景的几何结构与低质纹理；同时优化手与物体的姿态，实现准确的物体姿态估计。随后，进一步优化基于物理的渲染参数，包括粗糙度、镜面反射、反照率、手部可见性、皮肤反光及环境光照，以生成精确的反照率图和准确的手部光照与阴影区域。本方法在纹理重建上超越现有最优方法，据我们所知，是首个在物体纹理重建中考虑手物交互影响的工作。
原文摘要 · Abstract (English)
Reconstructing 3D models of dynamic, real-world objects with high-fidelity textures from monocular frame sequences has been a challenging problem in recent years. This difficulty stems from factors such as shadows, indirect illumination, and inaccurate object-pose estimations due to occluding hand-object interactions. To address these challenges, we propose a novel approach that predicts the hand's impact on environmental visibility and indirect illumination on the object's surface albedo. Our method first learns the geometry and low-fidelity texture of the object, hand, and background through composite rendering of radiance fields. Simultaneously, we optimize the hand and object poses to achieve accurate object-pose estimations. We then refine physics-based rendering parameters - including roughness, specularity, albedo, hand visibility, skin color reflections, and environmental illumination - to produce precise albedo, and accurate hand illumination and shadow regions. Our approach surpasses state-of-the-art methods in texture reconstruction and, to the best of our knowledge, is the first to account for hand-object interactions in object texture reconstruction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。