让两个人物按文本描述自然互动,生成更真实多样的动作序列。
Text2Interact: High-Fidelity and Diverse Text-to-Two-Person Interaction Generation
- 用大模型生成描述+单人动作库合成双人交互数据,无需额外采集。
- 保留文本中每个词的互动线索,使动作衔接更符合语义顺序。
- 适合需要复杂双人互动的虚拟角色、动画或游戏开发场景。
从文本建模人与人之间的互动仍具挑战性,因需同时具备逼真的个体动作和精确的时空协同。现有瓶颈在于:1)双人训练数据有限,难以覆盖多样互动细节;2)文本到动作建模粒度不足,语言条件常被压缩为单一句子嵌入。为此,我们提出Text2Interact框架,通过可扩展的高保真交互数据合成器与有效的时空协调流程,生成真实且与文本对齐的人类交互。首先,提出InterCompose合成方法,将大模型生成的交互描述与强单人动作先验对齐,基于提示和一人动作,检索候选动作,训练另一人反应生成器,并用神经动作评估器过滤低质量或错位样本,实现无额外采集的数据扩展。其次,提出InterActor模型,采用词级条件建模,保留动作启动、回应、接触顺序等词级线索,并引入自适应交互损失,强调上下文相关的双人关节配对,提升耦合性与物理合理性。大量实验显示,在动作多样性、保真度与泛化能力(包括分布外场景)上均有显著提升,用户研究验证其自然性。代码与模型将开源以促进复现。
原文摘要 · Abstract (English)
Modeling human-human interactions from text remains challenging because it requires not only realistic individual dynamics but also precise, text-consistent spatiotemporal coupling between agents. Currently, progress is hindered by 1) limited two-person training data, inadequate to capture the diverse intricacies of two-person interactions; and 2) insufficiently fine-grained text-to-interaction modeling, where language conditioning collapses rich, structured prompts into a single sentence embedding. To address these limitations, we propose our Text2Interact framework, designed to generate realistic, text-aligned human-human interactions through a scalable high-fidelity interaction data synthesizer and an effective spatiotemporal coordination pipeline. First, we present InterCompose, a scalable synthesis-by-composition pipeline that aligns LLM-generated interaction descriptions with strong single-person motion priors. Given a prompt and a motion for an agent, InterCompose retrieves candidate single-person motions, trains a conditional reaction generator for another agent, and uses a neural motion evaluator to filter weak or misaligned samples-expanding interaction coverage without extra capture. Second, we propose InterActor, a text-to-interaction model with word-level conditioning that preserves token-level cues (initiation, response, contact ordering) and an adaptive interaction loss that emphasizes contextually relevant inter-person joint pairs, improving coupling and physical plausibility for fine-grained interaction modeling. Extensive experiments show consistent gains in motion diversity, fidelity, and generalization, including out-of-distribution scenarios and user studies. We will release code and models to facilitate reproducibility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。