在线重建人体3D模型与世界轨迹，支持AR/VR实时交互
OnlineHMR: Video-based Online World-Grounded Human Mesh Recovery
- 采用双分支架构+因果缓存，实现逐帧流式推理
- 在EMDB和动态视频上性能媲美离线方法，支持在线处理
- 适用于增强现实、远程呈现等需要实时反馈的场景
从单目视频中恢复3D人体模型（HMR）是计算机视觉的重要任务。近年来，研究扩展到重建世界坐标系下的人体轨迹与运动，但多数方法仍为离线处理，依赖未来帧或全局优化，难以应用于需要实时反馈的交互场景，如AR/VR和远程呈现。为此，本文提出OnlineHMR，一个完全在线的框架，同时满足系统级因果性、忠实性、时序一致性和高效性四大核心要求。基于双分支架构，通过因果键值缓存设计和精选滑动窗口学习策略实现流式推理；同时引入以人为中心的增量式SLAM，在物理合理轨迹修正下实现在线世界对齐。实验表明，该方法在标准EMDB基准和高度动态自定义视频上性能接近现有分块式方法，且唯一支持在线处理。代码与页面见https://tsukasane.github.io/Video-OnlineHMR/
原文摘要 · Abstract (English)
Human mesh recovery (HMR) models 3D human body from monocular videos, with recent works extending it to world-coordinate human trajectory and motion reconstruction. However, most existing methods remain offline, relying on future frames or global optimization, which limits their applicability in interactive feedback and perception-action loop scenarios such as AR/VR and telepresence. To address this, we propose OnlineHMR, a fully online framework that jointly satisfies four essential criteria of online processing, including system-level causality, faithfulness, temporal consistency, and efficiency. Built upon a two-branch architecture, OnlineHMR enables streaming inference via a causal key-value cache design and a curated sliding-window learning strategy. Meanwhile, a human-centric incremental SLAM provides online world-grounded alignment under physically plausible trajectory correction. Experimental results show that our method achieves performance comparable to existing chunk-based approaches on the standard EMDB benchmark and highly dynamic custom videos, while uniquely supporting online processing. Page and code are available at https://tsukasane.github.io/Video-OnlineHMR/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。