arXiv:2604.21681cs.CV2026-04中稿 · ICLR被引 3

Sapiens2提升人类视觉任务的高保真与泛化能力,支持4K分辨率。

Sapiens2

论文配图:Sapiens2
图 1 · 摘自论文原文
  • 融合掩码图像重建与自蒸馏对比学习,统一预训练目标。
  • 在姿态、人体分割等任务上显著超越前代,4K模型提升明显。
  • 适合需要高精度人类图像理解的科研与工业应用。

我们提出Sapiens2,一类面向人类视觉的高分辨率Transformer模型家族,专注于泛化性、多功能性和高保真输出。模型规模从0.4亿到5亿参数不等,原生支持1K分辨率,层级变体支持4K。Sapiens2在预训练和微调阶段均显著优于前代。首先,为捕捉低层细节(用于密集预测)和高层语义(用于零样本或少样本场景),我们结合掩码图像重建与自蒸馏对比目标。评估表明,这一统一预训练目标更适合更广泛的下游任务。其次,在数据层面,我们在一个包含10亿张高质量人类图像的精选数据集上进行预训练,并提升了任务标注的质量与数量。第三,在架构上,引入前沿模型技术,支持更长训练周期并提高稳定性。4K模型采用窗口注意力机制以处理更长的空间上下文,并以2K输出分辨率预训练。Sapiens2达到新基准,在姿态识别(+4 mAP)、人体部位分割(+24.3 mIoU)、法向估计(角误差降低45.6%)等方面提升,并拓展至点云图与反照率估计等新任务。代码:https://github.com/facebookresearch/sapiens2

原文摘要 · Abstract (English)

We present Sapiens2, a model family of high-resolution transformers for human-centric vision focused on generalization, versatility, and high-fidelity outputs. Our model sizes range from 0.4 to 5 billion parameters, with native 1K resolution and hierarchical variants that support 4K. Sapiens2 substantially improves over its predecessor in both pretraining and post-training. First, to learn features that capture low-level details (for dense prediction) and high-level semantics (for zero-shot or few-label settings), we combine masked image reconstruction with self-distilled contrastive objectives. Our evaluations show that this unified pretraining objective is better suited for a wider range of downstream tasks. Second, along the data axis, we pretrain on a curated dataset of 1 billion high-quality human images and improve the quality and quantity of task annotations. Third, architecturally, we incorporate advances from frontier models that enable longer training schedules with improved stability. Our 4K models adopt windowed attention to reason over longer spatial context and are pretrained with 2K output resolution. Sapiens2 sets a new state-of-the-art and improves over the first generation on pose (+4 mAP), body-part segmentation (+24.3 mIoU), normal estimation (45.6% lower angular error) and extends to new tasks such as pointmap and albedo estimation. Code: https://github.com/facebookresearch/sapiens2

高分辨率人类视觉图像生成多任务学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。