用自回归模型自动补全2D人体姿态，提升3D重建效果
VARPose: Flexible 2D Pose Densification via Visual Autoregressive Modeling for Enhanced 3D Lifting
- 设计统一的离散编码器，适配不同密度的2D姿态
- 通过渐进式生成，从稀疏姿态预测更密集的关节位置
- 在未见姿态密度上表现良好，适合3D人体重建任务
视觉自回归建模（VAR）在自然图像生成中表现出色，但其在人体骨架等拓扑结构数据上的应用尚未探索。本文提出 VARPose，通过自适应补全2D稀疏姿态，增强3D姿态重建模型的可用信息。核心贡献有二：首先，提出无粒度依赖的姿态编码器（GPT），采用单一混合码本与残差量化策略，将不同密度的姿态编码为统一多尺度离散表示，具有强泛化能力；解码时冻结码本、仅重训练解码器，即可还原新粒度姿态。其次，提出 UniSkelar 统一自回归模型，将“关节密度”视为“尺度”，以粗到细方式，基于最稀疏姿态预测下一密度层级的词元序列。实验表明，VARPose不仅优于现有方法，且可推广至未见姿态密度，在3D姿态估计与人体网格恢复任务中带来显著性能提升。代码与模型已开源。
原文摘要 · Abstract (English)
Visual AutoRegressive Modeling (VAR) has excelled in natural image generation via next-scale prediction, but its use on topology-structured data like human skeletons is still unexplored. VARPose is proposed to adaptively densify 2D sparse poses, thereby enriching the anatomical information available for 3D lifting models. Our core contributions are twofold. First, we introduce a Granularity-agnostic Pose Tokenizer (GPT), which employs a single hybrid codebook and a residual quantization strategy to encode poses of varying densities into a unified, multi-scale discrete representation. Our results demonstrate the strong generalizability of this representation. By decoupling the representation from the projection, we can successfully decode novel pose granularities using a frozen codebook with a retrained decoder. Second, we propose UniSkelar, a unified autoregressive model that treats "joint density" as "scale". UniSkelar learns to predict the token sequence for the next density level in a coarse-to-fine manner, conditioned on the sparsest pose. VARPose not only outperforms state-of-the-art methods and generalizes to unseen granularities, but also confers tangible performance gains on downstream tasks, such as 3D Pose Estimation and Human Mesh Recovery, through 2D pose densification. Our code and model are available at https://github.com/BRL-SYSU/VARPose.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。