arXiv:2602.20556cs.CV2026-02

从野外视频中重建抗干扰的3D手部模型,提升真实场景下的表现。

WildGHand: Learning Anti-Perturbation Gaussian Hand Avatars from Monocular In-the-Wild Videos

  • 用动态扰动解耦模块分离时间变化的干扰因素。
  • 通过帧级加权掩码优化,实现空间与时间维度的干扰抑制。
  • 在复杂真实场景下性能显著超越现有方法,适合高保真手部建模应用。

尽管单目视频中3D手部重建取得进展,但多数方法依赖受控环境数据,在真实场景中因手物交互、极端姿态、光照变化和运动模糊等严重干扰而性能下降。为此,我们提出WildGHand,一种基于优化的框架,可在野外视频中自适应生成3D高斯点云手部化身。该框架包含两个关键组件:(i) 动态扰动解耦模块,将扰动作为优化过程中3D高斯属性的时间变化偏置显式表示;(ii) 扰动感知优化策略,生成每帧的非各向同性加权掩码以指导优化。二者协同实现跨时空维度的扰动识别与抑制。我们还构建了一个涵盖多样扰动的单目手部视频数据集,用于基准测试野外手部建模。在该数据集及两个公开数据集上的实验表明,WildGHand达到当前最优性能,相较基线模型在多个指标上显著提升(如PSNR相对提升达15.8%,LPIPS相对降低23.1%)。代码与数据集已开源。

原文摘要 · Abstract (English)

Despite recent progress in 3D hand reconstruction from monocular videos, most existing methods rely on data captured in well-controlled environments and therefore degrade in real-world settings with severe perturbations, such as hand-object interactions, extreme poses, illumination changes, and motion blur. To tackle these issues, we introduce WildGHand, an optimization-based framework that enables self-adaptive 3D Gaussian splatting on in-the-wild videos and produces high-fidelity hand avatars. WildGHand incorporates two key components: (i) a dynamic perturbation disentanglement module that explicitly represents perturbations as time-varying biases on 3D Gaussian attributes during optimization, and (ii) a perturbation-aware optimization strategy that generates per-frame anisotropic weighted masks to guide optimization. Together, these components allow the framework to identify and suppress perturbations across both spatial and temporal dimensions. We further curate a dataset of monocular hand videos captured under diverse perturbations to benchmark in-the-wild hand avatar reconstruction. Extensive experiments on this dataset and two public datasets demonstrate that WildGHand achieves state-of-the-art performance and substantially improves over its base model across multiple metrics (e.g., up to a $15.8\%$ relative gain in PSNR and a $23.1\%$ relative reduction in LPIPS). Our implementation and dataset are available at https://github.com/XuanHuang0/WildGHand.

3D手部建模高斯溅射野外视频抗干扰

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。