解决3D视觉中细丝状边界模糊难题,提升深度与新视角生成质量
Guardians of the Hair: Rescuing Soft Boundaries in Depth, Stereo, and Novel Views
- 通过图像抠图数据集构建训练流程,设计深度修复网络精准定位软边界
- 在单目深度、立体转视频和新视角合成任务中显著提升细部还原效果
- 适合关注高精度3D重建与真实感渲染的研究者与开发者
软边界(如细发丝)在自然与计算机生成图像中常见,但因前景与背景线索模糊混合,给3D视觉带来挑战。本文提出HairGuard框架,用于恢复3D视觉任务中的细粒度软边界细节。首先,设计基于图像抠图数据集的新颖数据整理流程,并引入深度修复网络自动识别软边界区域;该网络采用门控残差模块,在保持全局深度质量的同时,精确优化软边界附近深度。其次,在视图合成中,采用基于深度的前向映射保留高保真纹理,再通过生成式场景绘画器填补遮挡区域并消除软边界内冗余背景伪影;最后,颜色融合模块自适应结合映射与修复结果,生成几何一致且细节丰富的新视角图像。大量实验表明,HairGuard在单目深度估计、立体图像/视频转换及新视角合成任务中均达到当前最优性能,尤其在软边界区域提升显著。
原文摘要 · Abstract (English)
Soft boundaries, like thin hairs, are commonly observed in natural and computer-generated imagery, but they remain challenging for 3D vision due to the ambiguous mixing of foreground and background cues. This paper introduces Guardians of the Hair (HairGuard), a framework designed to recover fine-grained soft boundary details in 3D vision tasks. Specifically, we first propose a novel data curation pipeline that leverages image matting datasets for training and design a depth fixer network to automatically identify soft boundary regions. With a gated residual module, the depth fixer refines depth precisely around soft boundaries while maintaining global depth quality, allowing plug-and-play integration with state-of-the-art depth models. For view synthesis, we perform depth-based forward warping to retain high-fidelity textures, followed by a generative scene painter that fills disoccluded regions and eliminates redundant background artifacts within soft boundaries. Finally, a color fuser adaptively combines warped and inpainted results to produce novel views with consistent geometry and fine-grained details. Extensive experiments demonstrate that HairGuard achieves state-of-the-art performance across monocular depth estimation, stereo image/video conversion, and novel view synthesis, with significant improvements in soft boundary regions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。