通过标准化描述文本,提升动作-文字检索的准确性和泛化能力。
MoCHA: Denoising Caption Supervision for Motion-Text Retrieval
- 将每段描述投影到可从动作中还原的内容,减少语义噪声。
- 在HumanML3D和KIT-ML上分别提升3.1和10.3个百分点的检索精度。
- 支持无需大模型推理的轻量级版本,适合实际部署。
文本-动作检索系统通过对比学习从动作-描述对中学习共享嵌入空间。然而,每个描述并非确定性标签,而是有效描述的概率分布:不同标注者对同一动作生成不同文本,其中包含可从3D关节坐标推断的动作类型、身体部位、方向性等语义,也混杂了标注者风格和推断上下文等无法仅从动作数据恢复的信息。标准对比训练将每个描述视为唯一正样本,忽视了这种分布结构,导致同一动作内嵌入方差增大,削弱对齐效果。本文提出MoCHA,一种文本规范化框架,在编码前将每段描述投影至其可恢复内容,生成更紧凑的正样本簇和更清晰分离的嵌入。该方法具有通用性:即使使用确定性规则方法也能提升跨数据集迁移性能,而学习型规范化器带来更大收益。本文提出两种学习版本:基于LLM的GPT-5.2和无需推理时使用大模型的蒸馏FlanT5。MoCHA作为预处理步骤,兼容任意检索架构。应用于MoPa(MotionPatches)后,该方法在HumanML3D(H)和KIT-ML(K)上均达到新最优:LLM版本在H上实现13.9% T2M R@1(+3.1pp),在K上达24.3%(+10.3pp);无LLM的T5版本分别提升+2.5pp和+8.1pp。规范化使同一动作内文本-嵌入方差降低11%-19%,显著提升跨数据集迁移性能,其中H→K提升94%,K→H提升52%,表明标准化语言空间能获得更具迁移性的动作-语言表示。
原文摘要 · Abstract (English)
Text-motion retrieval systems learn shared embedding spaces from motion-caption pairs via contrastive objectives. However, each caption is not a deterministic label but a sample from a distribution of valid descriptions: different annotators produce different text for the same motion, mixing motion-recoverable semantics (action type, body parts, directionality) with annotator-specific style and inferred context that cannot be determined from 3D joint coordinates alone. Standard contrastive training treats each caption as the single positive target, overlooking this distributional structure and inducing within-motion embedding variance that weakens alignment. We propose MoCHA, a text canonicalization framework that reduces this variance by projecting each caption onto its motion-recoverable content prior to encoding, producing tighter positive clusters and better-separated embeddings. Canonicalization is a general principle: even deterministic rule-based methods improve cross-dataset transfer, though learned canonicalizers provide substantially larger gains. We present two learned variants: an LLM-based approach (GPT-5.2) and a distilled FlanT5 model requiring no LLM at inference time. MoCHA operates as a preprocessing step compatible with any retrieval architecture. Applied to MoPa (MotionPatches), MoCHA sets a new state of the art on both HumanML3D (H) and KIT-ML (K): the LLM variant achieves 13.9% T2M R@1 on H (+3.1pp) and 24.3% on K (+10.3pp), while the LLM-free T5 variant achieves gains of +2.5pp and +8.1pp. Canonicalization reduces within-motion text-embedding variance by 11-19% and improves cross-dataset transfer substantially, with H to K improving by 94% and K to H by 52%, demonstrating that standardizing the language space yields more transferable motion-language representations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。