用三点轨迹控制舞伴舞蹈,高效生成真实感双人舞动。
Dancing Points: Synthesizing Ballroom Dancing with Three-Point Inputs
- 仅需舞者三点轨迹输入,通过MLP网络直接预测跟舞者动作。
- 在球舞和LaFAN数据集上实现高保真动作生成,误差低于0.85米。
- 无需生成模型,适合实时沉浸式双人舞蹈应用开发。
华尔兹等交际舞是结构化但富有表现力的运动类型,其动作多样且舞伴间交互复杂,建模与生成极具挑战。本文证明,虚拟现实设备获取的三点轨迹可作为舞者动作的有效描述符,将全身体态交互简化为稀疏轨迹建模。由于维度低,我们采用高效的MLP网络,从领舞者三点输入直接预测跟舞者三点轨迹,解决了高维全身交互建模难题,并避免过拟合。通过利用动作固有结构并精心设计自回归过程,我们展示确定性神经网络可将三点轨迹转化为具身虚拟角色的动作,而该任务通常被认为欠约束,需依赖生成模型。此外,该方法在更大型、多样的数据集(如LaFAN)上表现稳健,具备良好泛化能力。本方法计算与数据效率高,为沉浸式双人舞蹈应用开辟新路径。代码与预训练模型见https://peizhuoli.github.io/dancing-points。
原文摘要 · Abstract (English)
Ballroom dancing is a structured yet expressive motion category. Its highly diverse movement and complex interactions between leader and follower dancers make the understanding and synthesis challenging. We demonstrate that the three-point trajectory available from a virtual reality (VR) device can effectively serve as a dancer's motion descriptor, simplifying the modeling and synthesis of interplay between dancers' full-body motions down to sparse trajectories. Thanks to the low dimensionality, we can employ an efficient MLP network to predict the follower's three-point trajectory directly from the leader's three-point input for certain types of ballroom dancing, addressing the challenge of modeling high-dimensional full-body interaction. It also prevents our method from overfitting thanks to its compact yet explicit representation. By leveraging the inherent structure of the movements and carefully planning the autoregressive procedure, we show a deterministic neural network is able to translate three-point trajectories into a virtual embodied avatar, which is typically considered under-constrained and requires generative models for common motions. In addition, we demonstrate this deterministic approach generalizes beyond small, structured datasets like ballroom dancing, and performs robustly on larger, more diverse datasets such as LaFAN. Our method provides a computationally- and data-efficient solution, opening new possibilities for immersive paired dancing applications. Code and pre-trained models for this paper are available at https://peizhuoli.github.io/dancing-points.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。