用文本到动作生成模型，零样本合成精准动作数据提升识别准确率。
Kinetic Mining in Context: Few-Shot Action Synthesis via Text-to-Motion Distillation
- 通过文本编码对应关系，将通用动作生成模型迁移到人体动作识别任务。
- 仅用每类10个样本，动作合成质量显著提升，识别准确率提高23.1个百分点。
- 适合需要少量标注数据的动作识别研究者，尤其关注数据增强与迁移学习。
大规模标注动作数据集的获取成本仍是基于骨骼的人体活动识别（HAR）的关键瓶颈。尽管文本到动作（T2M）生成模型提供了可扩展的合成数据来源，但其训练目标强调艺术性动作生成，且数据结构与HAR所需的运动学精确、类别可区分的动作存在根本差异，导致显著领域鸿沟，使通用T2M模型难以生成适用于HAR分类器的动作。为此，我们提出KineMIC（Kinetic Mining In Context），一种用于少样本动作合成的迁移学习框架。KineMIC通过假设文本编码空间中的语义对应关系可为运动学蒸馏提供软监督，利用CLIP文本嵌入建立稀疏HAR标签与T2M源数据之间的对应关系，指导微调，将通用T2M骨干网络转化为专门的少样本动作生成器。我们在HumanML3D作为源T2M数据集、NTU RGB+D 120子集作为目标HAR域的实验中验证了该方法，每类仅随机选取10个样本。结果表明，该方法生成的动作更连贯，提供强有力的增广数据，使分类准确率提升23.1个百分点。动画示例和补充材料见https://lucazzola.github.io/kinemic-page/。
原文摘要 · Abstract (English)
The acquisition cost for large, annotated motion datasets remains a critical bottleneck for skeletal-based Human Activity Recognition (HAR). Although Text-to-Motion (T2M) generative models offer a compelling, scalable source of synthetic data, their training objectives, which emphasize general artistic motion, and dataset structures fundamentally differ from HAR's requirements for kinematically precise, class-discriminative actions. This disparity creates a significant domain gap, making generalist T2M models ill-equipped for generating motions suitable for HAR classifiers. To address this challenge, we propose KineMIC (Kinetic Mining In Context), a transfer learning framework for few-shot action synthesis. KineMIC adapts a T2M diffusion model to an HAR domain by hypothesizing that semantic correspondences in the text encoding space can provide soft supervision for kinematic distillation. We operationalize this via a kinetic mining strategy that leverages CLIP text embeddings to establish correspondences between sparse HAR labels and T2M source data. This process guides fine-tuning, transforming the generalist T2M backbone into a specialized few-shot Action-to-Motion generator. We validate KineMIC using HumanML3D as the source T2M dataset and a subset of NTU RGB+D 120 as the target HAR domain, randomly selecting just 10 samples per action class. Our approach generates significantly more coherent motions, providing a robust data augmentation source that delivers a +23.1% accuracy points improvement. Animated illustrations and supplementary materials are available at https://lucazzola.github.io/kinemic-page/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。