用神经优化方法从多视角图像恢复人体三维形状与姿态。
HeatFormer: A Neural Optimizer for Multiview Human Mesh Recovery
- 提出HeatFormer，通过热图生成与对齐实现多视角人体参数迭代优化。
- 在多视角人体重建任务中表现优异，对遮挡和视角配置变化鲁棒。
- 适合用于养老、安防等固定多视角监控场景的人体行为建模。
我们提出一种新方法，可充分利用多个静态视角进行人体形状与姿态恢复，适用于安装于房间或开放空间角落的标定摄像头环境下的固定多视角人体监测（如老人照护、安全监控）。由于摄像头配置可能因环境而异，传统方法面临挑战。我们的核心思想是将该问题建模为神经优化过程。为此，我们设计了HeatFormer，一种基于新型变压器编码器-解码器的神经优化器，通过迭代优化SMPL参数实现多视角图像到人体模型的映射。其关键在于将人体参数估计转化为热图生成与对齐任务，从根本上摆脱视角配置依赖。我们在大量实验中验证了HeatFormer的有效性，包括精度、对遮挡的鲁棒性以及跨场景泛化能力。我们认为HeatFormer可在被动人体行为建模中发挥关键作用。
原文摘要 · Abstract (English)
We introduce a novel method for human shape and pose recovery that can fully leverage multiple static views. We target fixed-multiview people monitoring, including elderly care and safety monitoring, in which calibrated cameras can be installed at the corners of a room or an open space but whose configuration may vary depending on the environment. Our key idea is to formulate it as neural optimization. We achieve this with HeatFormer, a neural optimizer that iteratively refines the SMPL parameters given multiview images, which is fundamentally agonistic to the configuration of views. HeatFormer realizes this SMPL parameter estimation as heat map generation and alignment with a novel transformer encoder and decoder. We demonstrate the effectiveness of HeatFormer including its accuracy, robustness to occlusion, and generalizability through an extensive set of experiments. We believe HeatFormer can serve a key role in passive human behavior modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。