arXiv:2603.28766cs.CV2026-03被引 2

HandX构建了高精度双手动作数据集,推动真实手部交互生成。

HandX: Scaling Bimanual Motion and Interaction Generation

  • 采用解耦标注策略,结合大模型推理生成细粒度动作描述。
  • 在新数据集上训练的大模型生成更连贯的双手协同动作。
  • 适合研究手部精细运动、人机交互与生成模型的开发者。

人体动作合成进展迅速,但真实手部动作与双手协作仍研究不足。现有全身模型常忽略驱动灵巧行为的细微线索,如指节运动、接触时机与双侧协调。现有资源缺乏高保真双手序列,无法捕捉精细指动与协作动态。为此,我们提出HandX,一个涵盖数据、标注与评估的统一基础。整合并筛选现有数据集以提升质量,同时采集新动作捕捉数据,聚焦未充分覆盖的双手互动,并详细记录指部动态。为实现可扩展标注,引入解耦策略:先提取关键运动特征(如接触事件、指节屈曲),再利用大语言模型进行语义推理,生成与特征对齐的细粒度描述。基于此数据与标注,我们基准测试了扩散与自回归模型,支持多种条件输入。实验表明,生成动作质量高,且我们新提出的面向手部的指标有效支撑评估。进一步观察到显著的规模效应:在更大、更高质量数据上训练的更大模型,生成的双手动作更具语义一致性。数据集已公开,支持后续研究。

原文摘要 · Abstract (English)

Synthesizing human motion has advanced rapidly, yet realistic hand motion and bimanual interaction remain underexplored. Whole-body models often miss the fine-grained cues that drive dexterous behavior, finger articulation, contact timing, and inter-hand coordination, and existing resources lack high-fidelity bimanual sequences that capture nuanced finger dynamics and collaboration. To fill this gap, we present HandX, a unified foundation spanning data, annotation, and evaluation. We consolidate and filter existing datasets for quality, and collect a new motion-capture dataset targeting underrepresented bimanual interactions with detailed finger dynamics. For scalable annotation, we introduce a decoupled strategy that extracts representative motion features, e.g., contact events and finger flexion, and then leverages reasoning from large language models to produce fine-grained, semantically rich descriptions aligned with these features. Building on the resulting data and annotations, we benchmark diffusion and autoregressive models with versatile conditioning modes. Experiments demonstrate high-quality dexterous motion generation, supported by our newly proposed hand-focused metrics. We further observe clear scaling trends: larger models trained on larger, higher-quality datasets produce more semantically coherent bimanual motion. Our dataset is released to support future research.

动作生成双手协作数据集

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。