用一句话描述动作来预测人体3D运动，让模型更懂语义。
Zero-Gated Language-conditioned Human Motion Prediction
- 用视觉语言模型生成动作描述，转化为小量条件令牌注入网络。
- 在Human3.6M上比基线模型降低12.3%的平均关节误差（MPJPE）。
- 轻量设计，语言信息可选开启，适合需要语义理解的运动预测场景。
姿态历史是3D人体运动预测的核心动力学依据，但缺乏高层语义引导。本文提出ZGL，一种轻量级语言条件预测器，利用观察动作的描述作为语义先验，同时保留强健的姿态主干作为动态来源。仅渲染观测姿态，通过视觉语言模型生成一句描述，用冻结的CLIP-L文本塔编码，并投影为少量条件令牌。这些令牌通过零门交叉注意力适配器注入基于DCT的时空Transformer：每个适配器输出乘以一个初始化为零的可学习门控，使网络在初始化时与仅依赖姿态的基线完全等价，仅当语言信息能降低预测误差时才启用。在Human3.6M上，ZGL优于对比中的代表性运动预测基线；在CMUMocap上的结果进一步表明，紧凑的描述条件具备跨数据集迁移能力，为3D人体运动预测提供实用语义线索。
原文摘要 · Abstract (English)
Pose histories provide the core kinematic evidence for 3D human motion prediction, but they lack explicit high-level semantic guidance. This paper introduces ZGL, a lightweight language-conditioned predictor that uses captions of the observed motion as a semantic prior while preserving a strong motion backbone as the main source of dynamics. We render only the observed poses, generate a one-sentence description with a vision-language model, encode the caption with a frozen CLIP-L text tower, and project it into a small set of conditioning tokens. These tokens are injected into a DCT-based spatial-temporal Transformer by compact crossattention adapters with zero gates: each adapter output is multiplied by a learnable gate initialized to zero, so the full network is numerically identical to the pose-only baseline at initialization and can learn to use language only when it reduces prediction error. On Human3.6M, ZGL improves overall MPJPE over representative motion-prediction baselines in our comparison. Results on CMUMocap further show that compact caption conditioning transfers to a second benchmark and provides a practical semantic cue for 3D human motion prediction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。