修复姿态估计基准的标注错误，提升评估可靠性
Revisiting Reliability in the Reasoning-based Pose Estimation Benchmark
- 重新校准原始数据集图像索引，避免手动匹配错误
- 发现基准存在图像冗余、场景失衡等质量问题
- 开源修正后的标注，助力公平可复现的研究
基于推理的人体姿态估计（RPE）基准已成为评估姿态感知多模态大模型的广泛采用标准。然而，我们识别出关键的可复现性与基准质量问题，阻碍了公平一致的定量评估。最显著的问题是该基准使用了与原始3DPW数据集不同的图像索引，迫使研究者进行繁琐且易错的手动匹配以获取准确的真值（GT）标注（如MPJPE、PA-MPJPE）。此外，我们的分析揭示了若干固有的基准质量缺陷，包括显著的图像冗余、场景分布不均、姿态过于简单以及文本描述模糊，这些共同削弱了在多样化场景下的可靠评估。为减少人工负担并提升可复现性，我们通过细致的视觉比对，精心修正了真值标注，并公开发布这些修正后的标注作为开源资源，从而推动一致的定量评估，并促进未来人体姿态感知多模态推理的发展。
原文摘要 · Abstract (English)
The reasoning-based pose estimation (RPE) benchmark has emerged as a widely adopted evaluation standard for pose-aware multimodal large language models (MLLMs). Despite its significance, we identified critical reproducibility and benchmark-quality issues that hinder fair and consistent quantitative evaluations. Most notably, the benchmark utilizes different image indices from those of the original 3DPW dataset, forcing researchers into tedious and error-prone manual matching processes to obtain accurate ground-truth (GT) annotations for quantitative metrics (\eg, MPJPE, PA-MPJPE). Furthermore, our analysis reveals several inherent benchmark-quality limitations, including significant image redundancy, scenario imbalance, overly simplistic poses, and ambiguous textual descriptions, collectively undermining reliable evaluations across diverse scenarios. To alleviate manual effort and enhance reproducibility, we carefully refined the GT annotations through meticulous visual matching and publicly release these refined annotations as an open-source resource, thereby promoting consistent quantitative evaluations and facilitating future advancements in human pose-aware multimodal reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。