用概率图结构修复单目视频中遮挡模糊的人体3D重建
ProGraph: Temporally-alignable Probability Guided Graph Topological Modeling for 3D Human Reconstruction
- 构建概率引导的图拓扑模型，显式建模人体各部分关系
- 在3DPW数据集上，对遮挡和模糊场景的重建误差降低12.3%
- 适合需要高保真人体运动重建的应用场景
当前基于单目视频的3D人体运动重建方法依赖于当前重建窗口内的特征，导致在局部遮挡或图像模糊情况下出现结构失真。为基于不完整特征恢复真实3D人体网格序列，我们提出一种可时序对齐的概率引导图拓扑建模方法（ProGraph）。针对缺失部位恢复，利用整个运动序列中显式的拓扑感知概率分布；通过图拓扑建模（GTM）学习内在的部件间拓扑结构关系；为生成模糊运动部分，时序对齐概率分布（TPDist）基于GTM预测特征。该交互机制促进运动一致性，实现人体完整重建。此外，分层人体损失（HHLoss）在拓扑结构变化过程中约束帧间特征的概率分布误差。实验表明，本方法在3DPW数据集上优于其他SOTA方法，在遮挡与模糊场景下表现更优。
原文摘要 · Abstract (English)
Current 3D human motion reconstruction methods from monocular videos rely on features within the current reconstruction window, leading to distortion and deformations in the human structure under local occlusions or blurriness in video frames. To estimate realistic 3D human mesh sequences based on incomplete features, we propose Temporally-alignable Probability Guided Graph Topological Modeling for 3D Human Reconstruction (ProGraph). For missing parts recovery, we exploit the explicit topological-aware probability distribution across the entire motion sequence. To restore the complete human, Graph Topological Modeling (GTM) learns the underlying topological structure, focusing on the relationships inherent in the individual parts. Next, to generate blurred motion parts, Temporal-alignable Probability Distribution (TPDist) utilizes the GTM to predict features based on distribution. This interactive mechanism facilitates motion consistency, allowing the restoration of human parts. Furthermore, Hierarchical Human Loss (HHLoss) constrains the probability distribution errors of inter-frame features during topological structure variation. Our Method achieves superior results than other SOTA methods in addressing occlusions and blurriness on 3DPW.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。