用混合驱动+神经网络,让机械脸精准表达喜怒哀乐。
Morpheus: A Neural-driven Animatronic Face with Hybrid Actuation and Diverse Emotion Control
- 眼嘴用刚性结构控精准动作,鼻颊用绳索传微表情。
- 能从任意句子生成喜怒哀惧等情绪的细微控制信号。
- 适合做情感交互机器人或沉浸式剧场角色设计。
以往机械人脸因软硬件限制难以有效表达情绪。硬件上,传统刚性驱动虽精确但难在狭小空间布局,腱驱动虽省空间却难控;本文提出混合驱动:眼、口等关键部位用刚性结构实现高精度运动,鼻、颊等区域则用绳索传递微妙微表情。该设计使设备紧凑且功能丰富。算法上,引入自建模网络,通过梯度反向传播自动建立不同面部表情的blendshape系数与电机控制信号间的映射关系,并训练神经网络将语音输入转化为对应blendshape控制指令。由此可从任意语句生成幸福、恐惧、厌恶、愤怒等情绪的细腻表情输出,实现此前系统未达的精细化情绪控制。代码与硬件设计已开源。
原文摘要 · Abstract (English)
Previous animatronic faces struggle to express emotions effectively due to hardware and software limitations. On the hardware side, earlier approaches either use rigid-driven mechanisms, which provide precise control but are difficult to design within constrained spaces, or tendon-driven mechanisms, which are more space-efficient but challenging to control. In contrast, we propose a hybrid actuation approach that combines the best of both worlds. The eyes and mouth-key areas for emotional expression-are controlled using rigid mechanisms for precise movement, while the nose and cheek, which convey subtle facial microexpressions, are driven by strings. This design allows us to build a compact yet versatile hardware platform capable of expressing a wide range of emotions. On the algorithmic side, our method introduces a self-modeling network that maps motor actions to facial landmarks, allowing us to automatically establish the relationship between blendshape coefficients for different facial expressions and the corresponding motor control signals through gradient backpropagation. We then train a neural network to map speech input to corresponding blendshape controls. With our method, we can generate distinct emotional expressions such as happiness, fear, disgust, and anger, from any given sentence, each with nuanced, emotion-specific control signals-a feature that has not been demonstrated in earlier systems. We release the hardware design and code at https://github.com/ZZongzheng0918/Morpheus-Hardware and https://github.com/ZZongzheng0918/Morpheus-Software.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。