用扩散模型实现端到端的实时自拍视角全身动作重建
EgoForce: Robust Online Egocentric Motion Reconstruction via Diffusion Forcing
- 基于时序非对称噪声调度的扩散框架，逐帧去噪增量生成动作
- 在真实自拍场景下实现长时程全身动作重建，精度优于现有方法
- 适合需要低延迟、高鲁棒性的可穿戴设备与交互应用
随着具身智能体和增强现实设备的发展，自拍视角的观测数据已广泛可用，但其仅能间歇性捕捉手部信息，并依赖估计的头部轨迹。本文提出EgoForce，一种在线框架，用于从噪声大且稀疏的自拍输入中重建长期完整的身体动作。现有生成式方法虽能处理噪声和稀疏观测，但需固定长度的观察窗口，难以用于实时应用；而快速推理常依赖自回归预测，牺牲了鲁棒性。相比之下，我们采用受扩散强迫启发的时序非对称噪声调度的扩散方法，建模时间演化的不确定性，并随新流式观测逐步去噪。结合噪声鲁棒的插补策略，EgoForce在严格因果约束下逐步生成稳定连贯的全身动作。实验表明，该在线框架优于现有在线与离线方法，在具有挑战性的自拍场景中实现了长时程全身动作重建。
原文摘要 · Abstract (English)
With recent advances in embodied agents and AR devices, egocentric observations are readily available as input for real-world interactive online applications. However, egocentric viewpoints can only sporadically observe hands, in addition to the estimated head trajectory. We propose EgoForce, an online framework for reconstructing long-term full-body motion from noisy egocentric input. While existing generative frameworks can robustly handle noisy and sparse measurements, they assume a fixed-length observation window is available and are thus not suitable for real-time applications. Faster inference often relies on autoregressive prediction, sacrificing robustness. In contrast, we adopt a diffusion-based method with a temporally asymmetric noise schedule inspired by Diffusion Forcing. Specifically, our approach models temporally evolving uncertainty and incrementally denoises states as new streaming observations arrive. Combined with a noise-robust imputation strategy, EgoForce progressively generates stable and coherent full-body motion under strict causal constraints. Experiments demonstrate that our online framework outperforms existing online and offline methods, enabling long-horizon, full-body motion reconstruction in challenging egocentric scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。