arXiv:2409.10847cs.CLcs.CV2024-09被引 14

用随机排序保持序列结构,同时兼顾双向与顺序依赖的文本到动作生成新方法

BAD: Bidirectional Auto-regressive Diffusion for Text-to-Motion Generation

  • 通过随机排列实现序列污染,既保序又引入双向上下文
  • 在文本到动作生成任务上优于传统自回归和掩码模型
  • 适合需要精准时序与全局语义融合的动作生成场景

自回归模型因因果约束擅长建模序列依赖,但难以捕捉复杂双向模式;掩码模型虽能利用双向上下文,却常假设标记独立性,削弱序列建模能力。此外,掩码或吸收造成的序列破坏会引入不自然扭曲,影响学习。为此,我们提出双向自回归扩散(BAD),统一自回归与掩码模型的优势。BAD采用基于排列的污染技术,在保留自然序列结构的同时,通过随机排序强制因果依赖,有效捕获序列与双向关系。大量实验表明,BAD在文本到动作生成任务中优于自回归与掩码模型,为序列建模提供新预训练策略。代码已开源。

原文摘要 · Abstract (English)

Autoregressive models excel in modeling sequential dependencies by enforcing causal constraints, yet they struggle to capture complex bidirectional patterns due to their unidirectional nature. In contrast, mask-based models leverage bidirectional context, enabling richer dependency modeling. However, they often assume token independence during prediction, which undermines the modeling of sequential dependencies. Additionally, the corruption of sequences through masking or absorption can introduce unnatural distortions, complicating the learning process. To address these issues, we propose Bidirectional Autoregressive Diffusion (BAD), a novel approach that unifies the strengths of autoregressive and mask-based generative models. BAD utilizes a permutation-based corruption technique that preserves the natural sequence structure while enforcing causal dependencies through randomized ordering, enabling the effective capture of both sequential and bidirectional relationships. Comprehensive experiments show that BAD outperforms autoregressive and mask-based models in text-to-motion generation, suggesting a novel pre-training strategy for sequence modeling. The codebase for BAD is available on https://github.com/RohollahHS/BAD.

文本到动作扩散模型序列建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。