用哈尔变换提升动作预测,让模型同时看懂时空信息。
Multi-Resolution Haar Network: Enhancing human motion prediction via Haar transform
- 用二维哈尔变换将关节坐标升维,融合时空信息
- 在Human3.6M数据集上各时间步均优于现有方法
- 适合需要精准复杂动作预测的场景
3D人体姿态对现代计算机视觉与图形学至关重要,其预测旨在从历史序列中预判未来动作。现有方法因忽略人体运动在时间和空间轴上的任意性演变,导致在复杂动作(如随意摆姿或挥手)上表现不佳。为此,本文提出HaarMoDic网络,利用2D哈尔变换将关节映射至更高分辨率坐标系,使网络能同时获取时空信息。消融实验表明,多分辨率哈尔(MR-Haar)模块是关键组件:它不单独处理任一轴,而是通过2D哈尔变换将完整运动序列投影到混合坐标系,在不同分辨率下统一捕捉时空特征。该设计使网络可参考更广泛的信息。实验结果表明,HaarMoDic在Human3.6M数据集所有测试时间间隔上,均以更低的平均关节位置误差(MPJPE)超越现有最佳方法。
原文摘要 · Abstract (English)
The 3D human pose is vital for modern computer vision and computer graphics, and its prediction has drawn attention in recent years. 3D human pose prediction aims at forecasting a human's future motion from the previous sequence. Ignoring that the arbitrariness of human motion sequences has a firm origin in transition in both temporal and spatial axes limits the performance of state-of-the-art methods, leading them to struggle with making precise predictions on complex cases, e.g., arbitrarily posing or greeting. To alleviate this problem, a network called HaarMoDic is proposed in this paper, which utilizes the 2D Haar transform to project joints to higher resolution coordinates where the network can access spatial and temporal information simultaneously. An ablation study proves that the significant contributing module within the HaarModic Network is the Multi-Resolution Haar (MR-Haar) block. Instead of mining in one of two axes or extracting separately, the MR-Haar block projects whole motion sequences to a mixed-up coordinate in higher resolution with 2D Haar Transform, allowing the network to give scope to information from both axes in different resolutions. With the MR-Haar block, the HaarMoDic network can make predictions referring to a broader range of information. Experimental results demonstrate that HaarMoDic surpasses state-of-the-art methods in every testing interval on the Human3.6M dataset in the Mean Per Joint Position Error (MPJPE) metric.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。